-- Logs begin at Tue 2022-10-04 02:50:13 UTC, end at Tue 2022-10-04 03:30:51 UTC. -- Oct 04 03:14:57.121410 dv-cinder-849143-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 03:14:57.128799 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 03:14:57 2022] *** Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: nodename: dv-cinder-849143-9-iscsi Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: machine: x86_64 Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: clock source: unix Oct 04 03:14:57.130774 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: pcre jit disabled Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: detected number of CPU cores: 4 Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: current working directory: / Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: detected binary path: /usr/bin/uwsgi-core Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: your processes number limit is 47680 Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: your memory page size is 4096 bytes Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: detected max file descriptor number: 2048 Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: lock engine: pthread robust mutexes Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: thunder lock: enabled Oct 04 03:14:57.131235 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 03:14:57.132270 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 03:14:57.170817 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: Python main interpreter initialized at 0x557372f629c0 Oct 04 03:14:57.170817 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: python threads support enabled Oct 04 03:14:57.170817 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: your server socket listen backlog is limited to 100 connections Oct 04 03:14:57.170817 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: your mercy for graceful operations on workers is 90 seconds Oct 04 03:14:57.171248 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 03:14:57.171325 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: *** Operational MODE: preforking *** Oct 04 03:14:57.171569 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: *** uWSGI is running in multiple interpreter mode *** Oct 04 03:14:57.171569 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: spawned uWSGI master process (pid: 81366) Oct 04 03:14:57.171770 dv-cinder-849143-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 03:14:57.172517 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: spawned uWSGI worker 1 (pid: 81380, cores: 1) Oct 04 03:14:57.173777 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: spawned uWSGI worker 2 (pid: 81381, cores: 1) Oct 04 03:14:57.173777 dv-cinder-849143-9-iscsi devstack@keystone.service[81366]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 03:15:01.460815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 03:15:01.460815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: from cryptography.utils import int_from_bytes Oct 04 03:15:01.462191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 03:15:01.462191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: from cryptography.utils import int_from_bytes Oct 04 03:15:01.466124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 03:15:01.466124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: from cryptography.utils import int_from_bytes Oct 04 03:15:01.466593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 03:15:01.466593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: from cryptography.utils import int_from_bytes Oct 04 03:15:01.511571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 03:15:01.511766 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 03:15:01.511964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] command line args: [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 03:15:01.512239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 03:15:01.512424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ================================================================================ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 03:15:01.512683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] admin_token = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.512934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_dir = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.513125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.513312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_source = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.513496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] debug = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.513699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.514254 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.514254 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.514486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.514592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 03:15:01.514743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.514853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 03:15:01.514950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] command line args: [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 03:15:01.515053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515188 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_config_append = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 03:15:01.515405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ================================================================================ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 03:15:01.515607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_dir = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] admin_token = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.515905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_dir = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_source = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] debug = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.516950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517151 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_config_append = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.517996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_dir = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518495 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.518905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] publish_errors = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.519944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520603 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.520955 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] standard_threads = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.521886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_journal = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_json = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_stderr = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_syslog = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] publish_errors = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522839 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.522953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.523072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.523204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.523331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.523331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.524200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.524319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.524438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.524559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] standard_threads = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.524679 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.524793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.524919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.525034 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.525153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.525272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.525396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.525511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_journal = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.525511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_json = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.525740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.525860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_stderr = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.525981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526101 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_syslog = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.526221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 03:15:01.526461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.526928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527182 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.527997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.528986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.529955 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530772 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530884 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.530997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.531951 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532179 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532531 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.532953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533757 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.533999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.534884 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535475 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535796 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.535916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536598 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536712 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536943 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.536943 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.537933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.538974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539087 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539577 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539807 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.539919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540627 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.540968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541209 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541322 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.541973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.542908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543423 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.543877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.544941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.545986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546339 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546453 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.546880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.547978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.external = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.548905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549019 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.password = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.549959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.token = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.550823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.551989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552106 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.external = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.552657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553148 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553382 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.password = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553382 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.token = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.553972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.554922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.555910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556855 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.556970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557439 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.557903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558141 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.558968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.559888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.560845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561157 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561809 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.561931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562163 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.562981 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.563914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.564974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.565944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566249 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566365 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.566939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567445 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567793 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.567908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568033 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568163 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568727 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.568958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569772 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.569887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.570003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.570126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.570746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.570865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.570982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571642 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571757 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.571985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572109 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.572921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573266 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573380 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.573971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574092 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574450 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.574794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.575982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.576913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.577849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.578987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579109 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.579944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580297 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580411 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580883 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.580997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581466 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.581933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582279 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.582979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.583991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.584925 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585627 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.585936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.586555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.586555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.586791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.586908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.587968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588679 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.588918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.589908 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.590884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.591967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.592897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593488 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.593958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.594915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595677 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.595905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596146 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596260 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.596958 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.597998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598353 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.598937 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 03:15:01.599541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.599803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.600052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.600310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.600553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.600798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.601038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.601277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.601542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.601784 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.602032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.602272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.602519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.602757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.603008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.603274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.603523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.603756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.603999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.604236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 03:15:01.604409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 03:15:01.628350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.628585 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.628813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.630400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.630594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.630824 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.637728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.639620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.666204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.666437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.666684 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.666917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.720112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.720321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.768812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.769066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.778574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.778810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.924720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.924902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.924902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.925773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.948438 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.948630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.948714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.948867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.948947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.949948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.960751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.961526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.973781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974331 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.974984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.975107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.975204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.975381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 03:15:01.992277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.992487 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 03:15:01.992763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.992845 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.992978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 03:15:01.993058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 03:15:01.993283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.993370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.993516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 03:15:01.993596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.993798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.993878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.994056 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.994135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.994321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:01.994589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.003555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.003837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.004970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.011395 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.011952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.014310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.014597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.014829 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.014917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.015974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.016877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.017070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.017152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.017347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.017621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.040526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.041149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.041978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.042266 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.042622 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.042913 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.046748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.047038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.047464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.047575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.047862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.048151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.054000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.054287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.054734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.055025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.058228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.058523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.058814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.060640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.060794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.060941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061379 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.061950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.062102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.062432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.062659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.077205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.077492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.077967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.078251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.081325 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.082052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.084425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.084840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.085254 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.085557 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.089571 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.090239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.092745 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.093338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.095853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.096156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.096443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.096533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.096820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.096901 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097478 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097679 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.097963 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.113589 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.113881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.114079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.114162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.114376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.114457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.114651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.114920 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.121125 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.121786 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.122784 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.123365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.124340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.124620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.124905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.125189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.127339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.127627 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.127911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.127994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.128205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.128288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.128490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.128579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.128812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.129224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.140188 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.140476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.140763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.140920 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.141050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141215 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.141910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.142181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.142479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.142767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.157691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.158638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.160729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.161559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.163610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.164509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.165058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.166050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.166640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.166927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.167257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.167551 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.167841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.168250 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.173664 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.174583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.175187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.176121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.178308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.178599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.178889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.179187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.179303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.179692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.179987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.180274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.187046 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.187364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.187652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.188077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.188373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.188661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.194442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.194731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.195017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.195834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196033 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 03:15:02.196116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.196972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.197998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.198834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.199273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.199566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.199843 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 03:15:02.232375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557372f629c0 pid: 81381 (default app) Oct 04 03:15:02.233992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557372f629c0 pid: 81380 (default app) Oct 04 03:15:02.235782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa0c9-9e44-468a-b3df-d4ded2aa0ca8 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:02.236067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa0c9-9e44-468a-b3df-d4ded2aa0ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:02.236338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa0c9-9e44-468a-b3df-d4ded2aa0ca8 None None] PATH_INFO: `/v3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:02.240497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Tue Oct 4 03:15:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:08.993902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:08.994444 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:08.994444 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:08.998729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:09.004002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:09.004603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:09.004603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:09.076793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: INFO dbcounter [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] Registered counter for database keystone Oct 04 03:15:09.096142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG oslo_db.sqlalchemy.engines [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 03:15:09.102414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writer thread running {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 03:15:09.284889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG passlib.handlers.bcrypt [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81381) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 03:15:09.285199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG passlib.handlers.bcrypt [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81381) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 03:15:09.342922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:09.344548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:09.472877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 03:15:09.892201 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:09.962443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: INFO dbcounter [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] Registered counter for database keystone Oct 04 03:15:09.983796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG oslo_db.sqlalchemy.engines [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 03:15:09.989982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writer thread running {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 03:15:10.114340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd17580-0e16-44b2-a884-930be3d7b27a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:10.118676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:10.119337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:10.119622 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:10.119896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:10.127976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:10.179397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 03:15:10.179717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabe8d3-200e-4b75-84f4-0cc35fcfee29 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:10.188860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 4 03:15:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 03:15:10.194766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:10.211799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf257db-6750-41ce-8393-d949452df9e5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:10.216247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:10.217357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:10.217641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:10.217913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] PATH_INFO: `/v3/projects/0c792b1b39084f70bab95471acabbdd7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:10.219542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] RBAC: Authorizing `identity:get_project(project_id=0c792b1b39084f70bab95471acabbdd7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:10.271049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 03:15:10.271426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4af57e-40b1-4bb3-8c2a-465f10556a0a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:10.276395 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 4 03:15:10 2022] GET /identity/v3/projects/0c792b1b39084f70bab95471acabbdd7 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:12.379942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c799b1-7b7e-42ff-9e83-b55485d51bf7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:12.380344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c799b1-7b7e-42ff-9e83-b55485d51bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:12.380497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c799b1-7b7e-42ff-9e83-b55485d51bf7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:12.385247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:12.390689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:12.391071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:12.391267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:12.410902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:12.412449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:12.420199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 03:15:12.686747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c799b1-7b7e-42ff-9e83-b55485d51bf7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:12.698683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c799b1-7b7e-42ff-9e83-b55485d51bf7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:12.702859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:12.703489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:12.703773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:12.704047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:12.711640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:12.713388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3deaf33-f660-43f7-adf8-67dfbfaca76d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:12.721965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 4 03:15:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 03:15:12.727574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:12.736413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd689d-26ed-44d1-8cbe-a6e9f824db51 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:12.740715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:12.741331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:12.741612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:12.741889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:12.743789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:12.745592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f14dc-174d-416a-877a-8992fbc13336 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:12.747086 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Tue Oct 4 03:15:12 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:14.936292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76bcd6-30af-4c4d-8d14-e57e8db9ae12 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:14.936821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76bcd6-30af-4c4d-8d14-e57e8db9ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:14.936821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76bcd6-30af-4c4d-8d14-e57e8db9ae12 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:14.938163 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:14.943378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:14.943780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:14.943941 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:14.962910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f210-cedb-496a-859b-a97eac4e8e95 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:14.963088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:14.963294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f210-cedb-496a-859b-a97eac4e8e95 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:14.963497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f210-cedb-496a-859b-a97eac4e8e95 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:14.964174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:14.964764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:14.969893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:14.970179 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:14.970452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.011709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG passlib.handlers.bcrypt [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81380) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 03:15:15.012090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG passlib.handlers.bcrypt [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81380) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 03:15:15.044174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:15.060981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:15.062547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:15.160112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:15.308766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.333514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.415790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf26269-b9de-43b9-864c-92337b1654a1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.421873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.449749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.450379 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.450663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.450935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.462273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:15.471021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:15.472029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:15.473307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:15.478451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.522791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b75c8-c781-4aaa-afd8-f516670cc451 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.525747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.560267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.560422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.561136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.561349 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.561813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.580815 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d382-d5dd-4d18-b939-b3d3576774d6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.613991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.614594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.614880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.615172 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.615841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:15.616148 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:15.617590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:15.627902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:15.628903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:15.629410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:15.635156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.726680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 03:15:15.731047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ef50-4dc7-46bd-92cc-3f8892defabd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.732108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.734815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.764744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.765370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.765646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.765919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.767042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:15.768521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:15.770070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:15.775399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.791072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.832073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9ba66-c231-46a7-b150-378faf4ff7a1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.868116 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.868965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.869244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.869511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.875648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666ca4-6bac-49dd-81ce-24592471557b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:15.887886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:15.889350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:15:15.907544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:15.908537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:15:15.909317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:15:15.909317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:15:15.909317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:15.909477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:15.909477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:15.909610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:15:15.909683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:15.909683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:15.910212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:15.911697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:15.913215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:15.915029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.919102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:15.970805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:15.998784 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:16.016027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8ee9-f47e-4150-abad-a33ffbbf43b5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:16.016973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa47c8-271f-47bf-aade-fefb39dd1a48 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:16.049253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:16.049527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:16.049863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:16.050127 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:16.050211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:16.050407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:16.050496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:16.050680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:16.051058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:16.051397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:16.051493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.domain.id=default, user.id=3a6f18cf4a5b4b16ab604dd0d0188854) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:16.051716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:16.052789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f67d51-5409-45b8-9d26-a14692426920 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:16.053395 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547d3c4-bf5d-40e5-abfd-1e17d573e8aa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:16.059898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/role_assignments?user.id=3a6f18cf4a5b4b16ab604dd0d0188854&scope.domain.id=default&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 233 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:16.071438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:18.595076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834dc6f-a00c-4b53-80e5-bf5773f77883 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:18.595643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834dc6f-a00c-4b53-80e5-bf5773f77883 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:18.595643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834dc6f-a00c-4b53-80e5-bf5773f77883 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:18.596975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:18.602825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c308-0e99-4999-8119-6dc0ce049bc8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:18.603580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c308-0e99-4999-8119-6dc0ce049bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:18.604082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c308-0e99-4999-8119-6dc0ce049bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:18.631885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a396c308-0e99-4999-8119-6dc0ce049bc8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:18.633839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a396c308-0e99-4999-8119-6dc0ce049bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:18.719243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:18.739770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53ca9c-e917-47ed-8f65-2a5934b89df7 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:18.740265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53ca9c-e917-47ed-8f65-2a5934b89df7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:18.740708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53ca9c-e917-47ed-8f65-2a5934b89df7 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:18.742830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:18.748652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086eca-85b4-4ec3-9e32-7b94d3742878 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:18.749188 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086eca-85b4-4ec3-9e32-7b94d3742878 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:18.749644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086eca-85b4-4ec3-9e32-7b94d3742878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:18.768298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3a086eca-85b4-4ec3-9e32-7b94d3742878 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:18.769727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3a086eca-85b4-4ec3-9e32-7b94d3742878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:18.829891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:18.865145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:19.079230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360f04-baa6-4946-8287-81650e05fa1b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.079816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360f04-baa6-4946-8287-81650e05fa1b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.080261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360f04-baa6-4946-8287-81650e05fa1b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.081992 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 14/26] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:19.096418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.096949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.097442 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.121448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:19.122949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:19.146040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360f04-baa6-4946-8287-81650e05fa1b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.289708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:19.313110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360f04-baa6-4946-8287-81650e05fa1b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.360298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:19.360892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.361174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.361447 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.365969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.387723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:19.388452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:19.389250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:19.389991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:19.400136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.404320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:19.571443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088803b0-e06c-4adc-9bd9-e52f3a1a2235 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.576380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14dfe-4387-4455-954d-149307ef01d3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.579594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:19.609867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:19.610654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.610814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.611082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.623249 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:19.624907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8ddd29ab-ccf9-4584-913d-7f47f3c092c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:15:19.645375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:19.646431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:15:19.647382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:15:19.647382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:15:19.647382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:19.649852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 14/29] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:19.652658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07c16d-f1a7-432e-9e09-e7236d765c75 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.653414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:19.654516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.654933 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.655469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.656282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:19.656727 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:19.658260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81076205-4106-457a-9f81-d75d4a3d14da None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:19.658689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07c16d-f1a7-432e-9e09-e7236d765c75 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.659127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07c16d-f1a7-432e-9e09-e7236d765c75 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.661691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 15/30] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:19.662683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07c16d-f1a7-432e-9e09-e7236d765c75 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.677213 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:19.678693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.678978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.679287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.713841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:19.715040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:19.798373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:19.800741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07c16d-f1a7-432e-9e09-e7236d765c75 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.809767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 17/32] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:19.810950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.846815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:19.847705 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.848079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.848489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.849284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:19.849710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:19.851397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:19.865769 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 16/33] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:19.866633 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:19.890037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:19.943340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59faeb3-7ad7-42a1-ad94-7c2ce2d47bab None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.981707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:19.982688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.983299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.983819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.985214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:19.987060 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e28564-3490-4aac-a386-7fb5968e129e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:19.989021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 18/34] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:19 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:19.990175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6d7cd-50e8-465d-a47b-5bfc25d2fa3b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:19.990640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf47dc-de8a-47dd-8adc-7d6ea382ccac None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:19.991124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf47dc-de8a-47dd-8adc-7d6ea382ccac None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:19.991611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf47dc-de8a-47dd-8adc-7d6ea382ccac None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:19.993809 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 19/35] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:19.994805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf47dc-de8a-47dd-8adc-7d6ea382ccac None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:20.034008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.034636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.035170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.035345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.036257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.037633 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.067467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:20.110541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf47dc-de8a-47dd-8adc-7d6ea382ccac None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:20.144570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.145107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.145391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.145667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.146319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.147793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87808d38-f62f-47ad-988e-56edf430133b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.187200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 17/36] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 03:15:19 2022] POST /identity/v3/roles => generated 219 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:20.187200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:20.282363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:20.289059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 20/37] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:15:19 2022] POST /identity/v3/projects => generated 301 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:20.291028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.291344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.291625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.312828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7358-1083-42e8-aba3-0b0d979cc9cf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:20.330040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:20.334948 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:20.398570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.400072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.400072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.400414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.403099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.406342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.408483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 18/38] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:20.409341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:20.459559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:20.460481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:20.530730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:20.550106 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4e793-be51-4d03-88a9-d7534c465817 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:20.595303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaca52d-862e-45b8-91d1-8fa83f4a6ceb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:20.595481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.595941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.596232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.596577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.597268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.599591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a876-772a-4eeb-9537-a1d9a6af4b79 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.616294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 19/40] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 03:15:20 2022] POST /identity/v3/roles => generated 225 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:20.618423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424e02-ac82-4705-8edb-b739d9862ad5 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.618822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424e02-ac82-4705-8edb-b739d9862ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.619256 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424e02-ac82-4705-8edb-b739d9862ad5 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.620692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 20/41] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:20.628132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.628420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.628699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.643537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.644488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.644856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:20.644987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.645253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] PATH_INFO: `/v3/domains/default/users/3a6f18cf4a5b4b16ab604dd0d0188854/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.649104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3a6f18cf4a5b4b16ab604dd0d0188854, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.650684 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.656048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:20.658869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:20.673821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 22/42] 10.222.0.15 () {58 vars in 1361 bytes} [Tue Oct 4 03:15:20 2022] PUT /identity/v3/domains/default/users/3a6f18cf4a5b4b16ab604dd0d0188854/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:20.754245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:20.778415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:20.897415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47138-8ce7-40d9-acc7-83f3499ace4a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:20.905447 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:20.935298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:20.935916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:20.936207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:20.936487 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:20.937180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:20.938564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:20.955895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 23/44] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 03:15:20 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:21.030359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:21.139221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e392d7-f3a3-4d54-9c61-6257d7688572 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:21.175450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:21.176021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:21.176310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:21.176586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:21.177710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:21.180036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dac2db-2fb3-487a-aca6-a28157b7ec09 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:21.181788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 22/45] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:21.187229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:21.396198 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e209dd7-b7de-4b44-ab5f-9335bcfa96de None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:21.430049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:21.430596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:21.430881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:21.431180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:21.431844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:21.433300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0f4a3-6153-4fa8-af77-e1070eb60238 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:21.460421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:15:21 2022] POST /identity/v3/projects => generated 312 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:22.855387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708467c-6d42-475a-b3bd-89055cad744b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:22.855893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708467c-6d42-475a-b3bd-89055cad744b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:22.855973 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708467c-6d42-475a-b3bd-89055cad744b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:22.857282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:22.862624 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:22.862988 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:22.863202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:22.880848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:22.882036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:22.894704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:22.947703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:23.214694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708467c-6d42-475a-b3bd-89055cad744b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:23.218474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:23.316650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708467c-6d42-475a-b3bd-89055cad744b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:23.348679 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:23.349314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.349596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.349871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.361367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:23.368355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:23.369164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:23.369909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:23.375514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:23.474084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25ef9d-12dc-4c3f-a795-e21a6b7c27e7 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:23.508691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:23.509329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.509614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.509891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.510584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:23.510892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:23.512396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f676d-6900-47a0-8542-cd66d7370129 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:23.521801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 26/50] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:23.527879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:23.589594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:23.622261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19619-6b48-4c5f-9a5f-d0107da41129 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.622551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19619-6b48-4c5f-9a5f-d0107da41129 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.622842 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19619-6b48-4c5f-9a5f-d0107da41129 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.624233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 27/51] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:23.627656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232b79-df6f-41e6-95ba-0b1b3c07a832 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:23.629416 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.629710 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.630000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.647702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:23.649451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:23.652211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:23.663665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:23.664848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.665131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.665407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.666770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:23.668255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:23.669852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 25/52] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:23.675340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:23.716845 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 28/53] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:23.771189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fd0b-bb22-4ae0-9400-c35aa34b3664 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:23.799599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:23.805267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:23.805894 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.806181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.806461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.807615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:23.809095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:23.810811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 26/54] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:23.816536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:23.914366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c4e60-a73b-4e4f-9de4-1aa414b1a8d8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:23.948427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:23.949033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:23.949316 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:23.949654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:23.950389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.domain.id=default, user.id=3a6f18cf4a5b4b16ab604dd0d0188854) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:23.950702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:23.952192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:23.957850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 29/55] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/role_assignments?user.id=3a6f18cf4a5b4b16ab604dd0d0188854&scope.domain.id=default&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:23.987809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:24.095347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd76e6-f7d3-4f3f-8d31-3e5cbcd483fb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:24.130558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:24.131224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:24.131508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:24.131791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:24.132929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:24.134384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184542c8-8186-461f-bb04-2a02d581d6c2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:24.135988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 27/56] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:24.141636 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:24.146053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:24.245267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda5c91-9acd-4cec-8d99-7edb346e8950 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:24.278227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:24.278835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:24.279120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:24.279429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:24.280093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:24.281563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea40ad-9132-46e7-a5dc-4c8d9a37d2d5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:24.307557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 30/57] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 4 03:15:24 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:15:26.433288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3d06-39d6-45ba-a1bc-aa54c0ebf1c8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:26.433751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3d06-39d6-45ba-a1bc-aa54c0ebf1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:26.433838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3d06-39d6-45ba-a1bc-aa54c0ebf1c8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:26.435185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 28/58] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:26.440284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:26.440693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:26.440846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:26.458512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:26.459767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:26.508753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:26.526215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 31/59] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:26.792912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3d06-39d6-45ba-a1bc-aa54c0ebf1c8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:26.959941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:26.992546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3d06-39d6-45ba-a1bc-aa54c0ebf1c8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:27.024684 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:27.025312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:27.025597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:27.025877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:27.026985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:27.028430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6c213-e09e-4366-b4d7-3bbed3c04711 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:27.030145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 29/60] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:27.035681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:27.131605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef6523-6a7a-41d6-8200-870101cf4903 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:27.165203 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:27.165802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:27.166085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:27.166361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:27.167011 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:27.168481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb49aa-50ba-4426-90f2-c3bfe4e63130 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:27.193297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 32/61] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:27 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:30.421870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01ddb-faf1-48d8-a292-f0258c63f4ef None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.422380 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01ddb-faf1-48d8-a292-f0258c63f4ef None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.422451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01ddb-faf1-48d8-a292-f0258c63f4ef None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.423835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 30/62] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:30.430753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b76e87-e1b3-4cc3-a784-086e58496669 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.431624 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b76e87-e1b3-4cc3-a784-086e58496669 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.432076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b76e87-e1b3-4cc3-a784-086e58496669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.460292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b3b76e87-e1b3-4cc3-a784-086e58496669 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:30.461795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b3b76e87-e1b3-4cc3-a784-086e58496669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:30.507176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:30.550644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 33/63] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:30.602795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de450aa-4cbd-4c90-8f9a-87090e151233 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.603476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de450aa-4cbd-4c90-8f9a-87090e151233 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.603957 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de450aa-4cbd-4c90-8f9a-87090e151233 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.605710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 31/64] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:30.611207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.611512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.611800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.636257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:30.637398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:30.685911 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f95be-4660-4fb6-ab43-9e92fa7f5e01 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.686293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f95be-4660-4fb6-ab43-9e92fa7f5e01 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.686792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f95be-4660-4fb6-ab43-9e92fa7f5e01 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.688358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 32/65] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:30.693608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153f86f-dfdb-4b57-ae5d-d72e2b9efc7f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:30.693912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153f86f-dfdb-4b57-ae5d-d72e2b9efc7f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:30.694184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153f86f-dfdb-4b57-ae5d-d72e2b9efc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:30.721284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 34/66] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:30.736195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3153f86f-dfdb-4b57-ae5d-d72e2b9efc7f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:30.737518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3153f86f-dfdb-4b57-ae5d-d72e2b9efc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:30.830439 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 33/67] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:30.876473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.043196 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65fdc5-7a5e-4ded-9ce2-ce03b3250b41 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.046903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.101263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.102273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.102760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.103286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.104678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1374a-3d11-4ed0-883f-30107bcdbd6c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.105208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1374a-3d11-4ed0-883f-30107bcdbd6c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.105717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1374a-3d11-4ed0-883f-30107bcdbd6c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.107299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 34/68] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:31.113100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.113390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.113670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.118410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:31.123927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:31.124758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:31.125719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:31.126496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.136156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:31.137541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:31.218750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.228884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 35/70] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:31.229717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.251794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecd73a-805e-4504-9246-2f86d8dfb748 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.302881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.304145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.304674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.305214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.319197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:31.326462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:31.327328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:31.328083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:31.328680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 36/71] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:31.329474 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.337568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a151-687f-440e-adc4-2b57e7ea06d1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.355608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.377510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.377510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.377510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.377510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.377884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:31.377884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:31.378985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485afc2-94ea-4558-8222-74a0aea2e371 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:31.393175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 36/72] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:31.394385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0c533-e31e-4474-9d69-073fa2ada3a1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.394835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0c533-e31e-4474-9d69-073fa2ada3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.395318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0c533-e31e-4474-9d69-073fa2ada3a1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.396659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 37/73] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:31.397501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0c533-e31e-4474-9d69-073fa2ada3a1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.444088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f42b-f3d7-4e75-9aef-cbf33178d5e9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.462274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.480742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.481378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.481665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.481942 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.492131 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:31.497036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:31.497890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:31.498669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 37/74] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:31.499325 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.508604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0c533-e31e-4474-9d69-073fa2ada3a1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.544147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.544826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.545115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.545396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.546067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:31.546378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:31.547822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01521e77-53e4-4a8b-8380-94efa2f8a04f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:31.556443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:31.560628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.561181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.561489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.579314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:31.580502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:31.581045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.599292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f38b8-82d5-4b71-9b0a-2bc00ab194a6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.634895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.635566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.635853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.636134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.646480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 39/76] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:31.647682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.649428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:31.650916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:31.652726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 38/77] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:31.653418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.716929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.762131 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b8cd-3f7b-41ee-94c9-6b67024d54d4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.769801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade408a-e701-4872-91ab-d0ad2f840b7e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.797672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.798288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.798575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.798851 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.799552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:31.799870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:31.801268 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:31.805784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.806418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.806749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.807092 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.810147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 39/78] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:31.811161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.813887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.822195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:31.827479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:31.828528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 40/79] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:31.829811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.927340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b690a-fe1f-4f9a-a747-a7e926a9be13 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.935459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d3116-7e4f-41cb-8903-9ff6aaba2c6e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:31.946860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:31.964282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.964927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.965208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.965491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:31.966635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:31.968122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:31.969709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 40/80] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:31.970471 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:31.975541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:31.976186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:31.976470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:31.976749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.048701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.049795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.050169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.082387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2097-d101-407e-9ef7-a30cacad99c5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.115969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.116630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.116919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.117201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.118334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.119826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.121398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.122165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.152241 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 41/82] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:32.153085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.164801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.225539 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beae3-e5a8-44ee-9a75-85d1fcb765cc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.252866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5442-848e-41ce-ab0b-6fe6db9b8afb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.258741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.259378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.259688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.259972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.260639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:32.260959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.262350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.267324 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.272233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 42/83] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.273265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.291489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.292131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.292415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.292701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.293828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.295321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.297043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 42/84] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.299968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.372387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4034e74-e71b-4ec8-be0c-e7978876e119 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.382969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.405624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c61304-024c-4240-98ba-8e94d95ed6d5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.407334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.407951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.408238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.408519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.438841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.439481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.439768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.440052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.471964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.475385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.477608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.507333 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.508823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.573235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 43/85] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:32.574068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.586262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.608096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 43/86] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:32.608917 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.669686 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.672417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36923d6-d85b-49ad-8f3a-fb8708807c13 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.705726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.706348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.706637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.706920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.707544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d037a-8749-4fab-b462-0e9f30370093 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.708100 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.709562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.711778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.712534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.742362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.742998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.743310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.743595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.744725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.746156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.752029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.752029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.793403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:32.813190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a8c7-67f8-4a3a-8f91-44a459490551 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.846187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.846331 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8fb72-91b6-419c-95c4-1007c49b0271 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:32.846798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.847076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.847379 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.848036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.849463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.871492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 45/89] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:32 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:32.872323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.882098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:32.882808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:32.883102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:32.883406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:32.884629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:32.886129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:32.887941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:32.888754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:32.919659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.007068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32cd6-2d55-45bf-b93b-9554d3b7e6d6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.008817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd23ea-46d8-4396-90e6-853769b96b56 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.023809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.045762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.046450 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.046895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.047352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.048729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.049589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.050208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.050303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.051298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.051620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.052963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:33.053068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:33.053068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.053464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:33.059765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.065478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:32 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=800a18a269264faba7cbec11735ac79b => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:33.067339 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:33.182171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.195712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b036f-d733-489e-b420-0aa0ea2f02d7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.210325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78346458-626c-4d25-90cb-694ad956c8a0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.239092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.239757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.240043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.240320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.256789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.257652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.257913 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.258200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.259384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:33.259986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.261465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.268845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:33 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:33.270086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:33.331492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.336173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.338248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.411658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832a5c-951a-4fbb-9434-47a76ebd0a40 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.452554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.453188 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.453521 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.453874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.455016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=3a6f18cf4a5b4b16ab604dd0d0188854) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:33.455016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.456456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.463599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:33 2022] GET /identity/v3/role_assignments?user.id=3a6f18cf4a5b4b16ab604dd0d0188854&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:33.479578 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.487643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 47/95] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:33.494096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:33.639529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707d84-636e-4bfb-aabe-bdd0e47727a6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.648853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:33.690894 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.691568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.691855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.692135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] PATH_INFO: `/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.695882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] RBAC: Authorizing `identity:get_project(project_id=f518bfe4dc6f4a00abda405cd6bd6960)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.697343 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ce7c0-224e-4b77-a924-15fdbc0c1f9c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.699313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:33 2022] GET /identity/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:33.707089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:33.851936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed03a-09f4-4569-8a01-d5614eaf66a2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:33.908568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:33.909124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:33.909409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:33.909694 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:33.910448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=f518bfe4dc6f4a00abda405cd6bd6960, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:33.910767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:33.912289 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c04e9-e0c8-4034-b68e-72a72075c127 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:33.923486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:33 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=f518bfe4dc6f4a00abda405cd6bd6960&role.id=800a18a269264faba7cbec11735ac79b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:35.963342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9bae1-4557-406d-98d8-d8a10ad32b10 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:35.963850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9bae1-4557-406d-98d8-d8a10ad32b10 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:35.963951 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9bae1-4557-406d-98d8-d8a10ad32b10 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:35.965240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:35.981362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:35.981899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:35.981899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.012723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:36.013938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:36.060923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36172-48d9-4181-887b-3cbb2e6b6c81 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.061226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36172-48d9-4181-887b-3cbb2e6b6c81 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.061556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36172-48d9-4181-887b-3cbb2e6b6c81 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.062915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 51/99] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:36.068296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.068668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.068950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.095900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:36.104130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:36.105359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:36.121398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 49/100] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:36.231107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 52/101] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:36.443913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:36.563256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1475b4-98e0-414c-b331-0497a9a1932c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:36.598322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:36.598968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.599287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.599850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.601107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:36.602588 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d8d61-cf0c-494f-8768-b47f2257913f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:36.604227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 50/102] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:36.609814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:36.666912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47360cb7-e407-4b38-9bc7-1f05fc3a47fd None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.667292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47360cb7-e407-4b38-9bc7-1f05fc3a47fd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.667512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47360cb7-e407-4b38-9bc7-1f05fc3a47fd None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.668861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 51/103] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:36.674146 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.674439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.674720 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.704181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:36.705417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:36.789580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdea58-6f15-457c-b523-75bf0d63bba8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:36.794378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:36.805494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:36.836623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 52/104] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:36.837520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:36.859051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:36.860159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.860588 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.861038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.861886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:36.863683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb05b97-efb9-4752-8328-281ab31bbf3d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:36.932174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 53/105] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:15:36 2022] POST /identity/v3/projects => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:36.933657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b93d-7393-4c96-b11d-33e197c54319 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.934102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b93d-7393-4c96-b11d-33e197c54319 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.934511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b93d-7393-4c96-b11d-33e197c54319 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.940707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 54/106] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:36.952697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e283a7-0c5c-4a38-9ca6-5f7ffc1205ba None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:36.953212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e283a7-0c5c-4a38-9ca6-5f7ffc1205ba None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:36.953634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e283a7-0c5c-4a38-9ca6-5f7ffc1205ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:36.955072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccadc4f-5d61-4964-b8c8-78b49c4f2030 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:36.989941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-89e283a7-0c5c-4a38-9ca6-5f7ffc1205ba None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:36.991981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-89e283a7-0c5c-4a38-9ca6-5f7ffc1205ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:37.009029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.009851 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.010240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.010685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.037420 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-daf96ed2-5937-44e4-a212-e96452d35c9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:37.049405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:37.050321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:37.051078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 53/107] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:37.055715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc5a5f-3689-41fe-b2ae-ba7746f38139 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.059351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc5a5f-3689-41fe-b2ae-ba7746f38139 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.059651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc5a5f-3689-41fe-b2ae-ba7746f38139 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.063910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 54/108] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:37.064821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc5a5f-3689-41fe-b2ae-ba7746f38139 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.151026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 55/109] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:37.152624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.152916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.153197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.172518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.184235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:37.185932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:37.245904 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc5a5f-3689-41fe-b2ae-ba7746f38139 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.281441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 56/110] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:37.283446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.301177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.322432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.323068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.323378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.323653 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.324336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:37.324649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:37.326202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346a9a0-7bf1-4da5-87e7-5d7aa7ec4fc1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:37.345518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 55/111] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:37.349282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698e7e-3142-41fb-a654-adb8c059fc04 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.351736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698e7e-3142-41fb-a654-adb8c059fc04 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.352019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698e7e-3142-41fb-a654-adb8c059fc04 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.354674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 56/112] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:37.355845 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d698e7e-3142-41fb-a654-adb8c059fc04 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.446924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcea9e-fa26-49e7-af31-f0b9b24f0388 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.492430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.493990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.494501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.495338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.507708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d698e7e-3142-41fb-a654-adb8c059fc04 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.508431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:37.515627 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3f930c25-a983-4487-b098-41e8a501db7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:37.515980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:37.516806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:37.517533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:37.522447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 57/113] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:37.524448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.524879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.525302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.546398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:37.548045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:37.551298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.552227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.552441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.552787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.554810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:37.556448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:37.558283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 57/114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:37.559135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.627214 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 58/115] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:37.628158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.636482 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.699392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37557-df67-4562-99a8-025a5aa0819e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.740108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb21380-7940-4699-9d49-f65a709c114f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.745737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.746380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.746663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.746939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.759914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:37.766699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:37.767774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 58/116] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:37.768789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.778353 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.779128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.779715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.780092 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.780873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:37.781338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:37.783133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:37.793334 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 59/117] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:37.794441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.812127 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.883181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:37.885577 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c9a9-8885-4ec2-92f2-4af76f295f89 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.921832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8c1fd-f336-452f-8786-e04b1372abb4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:37.922843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.923635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.924028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.924309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.925448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:37.926907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:37.933838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 59/118] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:37.933838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:37.962384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:37.962939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:37.963305 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:37.963585 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:37.973937 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:37.978870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:37.979918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 60/119] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:37.980846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.049849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149e4f1-4201-4439-a62a-2b7a96552e40 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.051632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.088069 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.088685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.088970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.089246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.089913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:38.090222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.091678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.100241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 60/120] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.101581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.107670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166b48-c729-4925-b1dd-db4ecd44e95c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.137992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.146919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.147594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.148549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.148833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] PATH_INFO: `/v3/users/14f88fa754624898b5c00bb859c3ef01` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.165619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] RBAC: Authorizing `identity:get_user(user_id=14f88fa754624898b5c00bb859c3ef01)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.167306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.169257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 61/121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:37 2022] GET /identity/v3/users/14f88fa754624898b5c00bb859c3ef01 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.170016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.207123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bb9c-4e8c-4c2c-b841-fa0e69b04e5e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.241004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.241622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.242180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.252560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:38.258282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:38.259165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:38.259966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 61/122] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:38.260576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.278011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95ad7-1a27-44be-888c-22f935899408 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.286248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.313486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.314161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.314446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.314726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/1b9cf77b90164c2b900d5824c6606349/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.317024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, user_id=1b9cf77b90164c2b900d5824c6606349, role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.318485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.347172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 62/123] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:38 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/1b9cf77b90164c2b900d5824c6606349/roles/4363620f6a3647d0a382928ffef15ec7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:38.347172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.371546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bdd95-5245-4901-86fb-c18f2cb5d165 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.418554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.443990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.445458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.446425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.446735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.447576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:38.448170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.449915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.466442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 62/124] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.467896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.491371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0307f3-bc6b-4c23-864b-9cf70d50083a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.531458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.532213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.532501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.532819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.534986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.536692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.538591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 63/125] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.539626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.587094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.607909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a915b90-ba81-474e-867a-b3ab26583d5c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.654903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74677eb5-30fe-4606-a9fb-a5208fa54e52 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.657885 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.658594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.658974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.660322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.674563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.697054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.697931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.698500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.698980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.699918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:38.700447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.702054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.711448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.712558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.749724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.751167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.827371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce024f77-0c03-4474-b0c4-85b65052cf9c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:38.863082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:38.865476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:38.866130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:38.866564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:38.866895 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:38.868589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:38.870072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:38.876296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 65/127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:38.876296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.889718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 63/128] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:38.890978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:38.934023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.002306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29b37a-b922-4659-8462-1307061f26b1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.016096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfd360-6fa2-4b50-8183-17867af68c9f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.039207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.039860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.040147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.040427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.041583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.043422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.045114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 66/129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.045854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.062698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.063878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.064168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.064447 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.066407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.067921 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.070119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 64/130] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:38 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.070901 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.134257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.192079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e453c-a6d2-4143-aa12-63a86128d25d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.196077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6404b-e024-4792-8103-5c43afd05eff None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.206445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.231012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.231707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.231994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.232370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] PATH_INFO: `/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.233519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] RBAC: Authorizing `identity:get_project(project_id=f518bfe4dc6f4a00abda405cd6bd6960)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.234866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.241110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 67/131] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:39 2022] GET /identity/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.241110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.250394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.251398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.251831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.252260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.254717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.257346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.260441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 65/132] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:39 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.261542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.377053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef2799-d4a7-4a88-a6c0-d66b7b479511 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.388849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87309e-7a7c-4381-9fc6-28dcceb9ad69 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.402153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.414738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.415407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.415693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.415975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.417100 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.418556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.420218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 66/133] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:39 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.420959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.433742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.434420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.434707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.435398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/1b9cf77b90164c2b900d5824c6606349/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.437402 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, user_id=1b9cf77b90164c2b900d5824c6606349, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.438833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.452794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 68/134] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:39 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/1b9cf77b90164c2b900d5824c6606349/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:39.453881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.503529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.573835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be138-27fe-40bd-97e6-df1f24aed4cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.576710 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd87dbb-941a-493f-9eb6-6dd6b75a0131 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.616617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.617791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.618256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.618739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] PATH_INFO: `/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960/users/1b9cf77b90164c2b900d5824c6606349/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.621036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f518bfe4dc6f4a00abda405cd6bd6960, user_id=1b9cf77b90164c2b900d5824c6606349, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.622742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.622945 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ac6d-a272-4d49-b841-805aef98d9bc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.623515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.623866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.624231 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.625024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=14f88fa754624898b5c00bb859c3ef01) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:39.625410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.626926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef135-3a0a-42be-ad3a-0539ed5cbb2b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.641125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 69/135] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:39 2022] GET /identity/v3/role_assignments?user.id=14f88fa754624898b5c00bb859c3ef01&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=6c219595326a412e8f909d29c07e8c6f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:39.642200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab6b2-89aa-4894-a8c3-a8ed100dc029 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.642495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab6b2-89aa-4894-a8c3-a8ed100dc029 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.642781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab6b2-89aa-4894-a8c3-a8ed100dc029 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.644323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 70/136] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:39.645042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bab6b2-89aa-4894-a8c3-a8ed100dc029 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:39.650344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 67/137] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:39 2022] PUT /identity/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960/users/1b9cf77b90164c2b900d5824c6606349/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:39.653580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.654210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.654683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.678584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:39.680624 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:39.742483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.776560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:39.780344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bab6b2-89aa-4894-a8c3-a8ed100dc029 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:39.783254 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:39.822006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:39.822929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:39.823251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:39.823540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/3a6f18cf4a5b4b16ab604dd0d0188854/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:39.825647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, user_id=3a6f18cf4a5b4b16ab604dd0d0188854, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:39.827053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:39.841809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 71/139] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:39 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/3a6f18cf4a5b4b16ab604dd0d0188854/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:40.154305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:40.274828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1d9f0-99aa-4e8a-99f5-43c52d790263 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:40.342753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:40.343699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:40.344123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:40.344537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:40.345854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:40.347481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afece2-a128-46c2-8e56-73233d49ff04 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:40.349532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 69/140] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:40.355094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:40.514127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0e853-f3d2-4426-8bc5-19ca5f17099e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:40.564861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:40.565496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:40.565788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:40.566069 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:40.566760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:40.568255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62859809-3228-42ae-b1aa-caea19bc9e22 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:40.616532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 72/141] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:40 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:41.135903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ead2-e5cb-4461-83a0-0d1b691a1c07 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:41.136831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ead2-e5cb-4461-83a0-0d1b691a1c07 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:41.137525 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ead2-e5cb-4461-83a0-0d1b691a1c07 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:41.140055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 70/142] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:41.147999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:41.148382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:41.148600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:41.174169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:41.175426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:41.191497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:41.262513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 73/143] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:41.620541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ead2-e5cb-4461-83a0-0d1b691a1c07 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:41.686776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:41.748618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ead2-e5cb-4461-83a0-0d1b691a1c07 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:41.792706 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:41.793403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:41.793689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:41.793966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:41.807623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:41.812601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:41.813464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:41.814302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 71/144] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:41.824002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:41.947248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172a982-5840-4210-ae56-c93d377bb68f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:41.986080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:41.986956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:41.987450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:41.987877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:41.988860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:41.989332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:41.990877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:42.005409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 74/145] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:42.013323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:42.151675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c68c9-cc16-43bc-b1ee-4a66ef4f7f18 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:42.161658 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:42.199943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:42.200512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:42.200794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:42.201070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:42.202286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:42.203903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:42.205666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 72/146] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:42 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:42.211201 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:42.241660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:42.350031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514994a4-8c47-4eae-953b-3854b726de28 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:42.395858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:42.396802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:42.397284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:42.398348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:42.476648 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:42.478419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:42.494434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:42.603640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 75/147] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:42.609323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:42.728407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9450-a437-4d9b-98aa-c2c25b018984 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:42.772232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:42.772952 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:42.773240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:42.773520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:42.774694 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:42.776200 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1974c1-19c0-4a1e-9345-cea7beef5b99 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:42.777882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 73/148] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:42 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:42.784243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:42.896681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ebd0-2e70-430e-bba7-d300cfd9038e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:42.911349 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:42.936965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:42.937590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:42.937873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:42.938150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:42.938902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:42.939267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:42.940792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072f6c5-0644-431e-b5ed-af70657c6de7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:42.947944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 76/149] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:42 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:43.124232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0536417-93cc-4042-92a9-b3668a67efe0 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.124628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0536417-93cc-4042-92a9-b3668a67efe0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.124817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0536417-93cc-4042-92a9-b3668a67efe0 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.126542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 74/150] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:43.132130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51670a-f713-4f63-b8df-79354664f23f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.132441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51670a-f713-4f63-b8df-79354664f23f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.132704 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51670a-f713-4f63-b8df-79354664f23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.156095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dc51670a-f713-4f63-b8df-79354664f23f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:43.157639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dc51670a-f713-4f63-b8df-79354664f23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:43.241938 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 77/151] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:43.262424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5075-11a7-467f-9756-02788ced3329 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.263035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5075-11a7-467f-9756-02788ced3329 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.263548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5075-11a7-467f-9756-02788ced3329 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.265136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 75/152] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:43.270778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.271331 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.271801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.292677 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:43.294176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:43.368587 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 78/153] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:43.783423 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f270b-5556-4f87-9f61-83db8d930f10 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.784455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f270b-5556-4f87-9f61-83db8d930f10 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.784964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f270b-5556-4f87-9f61-83db8d930f10 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.786514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 76/154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:43.787829 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:43.792100 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c616-6193-47d3-a951-257746a0a794 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.792641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c616-6193-47d3-a951-257746a0a794 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.793137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c616-6193-47d3-a951-257746a0a794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.831553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ec76c616-6193-47d3-a951-257746a0a794 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:43.834244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ec76c616-6193-47d3-a951-257746a0a794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:43.879422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:43.922475 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:43.977391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 77/155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:43.978665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc480ca-cfed-4ae5-b837-b92e2a8ee578 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:43.978952 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc480ca-cfed-4ae5-b837-b92e2a8ee578 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:43.979255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc480ca-cfed-4ae5-b837-b92e2a8ee578 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:43.980646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 78/156] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:43.981202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc480ca-cfed-4ae5-b837-b92e2a8ee578 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.007282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcb7a8-3eb1-4568-934e-113758fc15b8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.087798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.088443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.088729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.089002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.105310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-51420607-f54b-4645-af6d-9e8b857372c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:44.110390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:44.111255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:44.112008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:44.123717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 79/157] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:44.125423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.125711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.125995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.159932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:44.164458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:44.259826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc480ca-cfed-4ae5-b837-b92e2a8ee578 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.307778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 80/158] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:44.309090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.343958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.344527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.344812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.345088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.359510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:44.392358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:44.397601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:44.398607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:44.399419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:44.402852 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 79/159] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:44.403909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.463882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:44.509996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ca55a-fb25-4ca7-8c0b-8f40bba08b35 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.567171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ede28-ce7d-449f-b636-90769a30c6dd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.610160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.610716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.610987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.611352 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.611959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:44.615165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:44.615165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:44.627871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.628818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.629319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.629794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.631893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:44.632758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.645349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:44.659311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:44.660272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:44.661030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:44.663477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 80/161] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:44.664589 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.778661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86b52-4ff6-4b24-bb6b-79bf90319202 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.781962 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:44.815364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.816807 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.817288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.817767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.819125 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:44.820877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:44.822687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 81/162] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:44.823782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:44.863599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:44.898855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd905e-8569-4ad5-a1ab-7bc83049847c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.928512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b85e1-5a97-4ca2-b7ff-a9d711cacc7f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:44.968379 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:44.969587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:44.970091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:44.970607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:44.971506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:44.972039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:44.973788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:44.983236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 82/163] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:44.984441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.008119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.008743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.009189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.009611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.010418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:45.010937 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.012507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.015480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.032414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 82/164] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.033315 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.093780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3db1a-52d9-4956-af28-863c938f3aea None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.147924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8329-0c82-4608-8635-0040247a887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.148928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8329-0c82-4608-8635-0040247a887c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.150256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8329-0c82-4608-8635-0040247a887c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.150769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8329-0c82-4608-8635-0040247a887c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.238027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07867bfe-da7a-4233-8ff8-69e5b78b6faf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.295277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d8329-0c82-4608-8635-0040247a887c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.298554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d8329-0c82-4608-8635-0040247a887c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.303725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.304718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.305231 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.305745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.317342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:45.323262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:45.324170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:45.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:45.325530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.327461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 83/165] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:45.328494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.342782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.414763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 83/166] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:45.418900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8329-0c82-4608-8635-0040247a887c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.443753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9646b-47c7-45ac-9829-70fce81ab194 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.478898 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.479573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.479861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.480140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] PATH_INFO: `/v3/users/14f88fa754624898b5c00bb859c3ef01` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.481284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] RBAC: Authorizing `identity:get_user(user_id=14f88fa754624898b5c00bb859c3ef01)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.482765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6dd799-e955-48a3-8f59-502a207fbbf7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.488761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 84/167] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/users/14f88fa754624898b5c00bb859c3ef01 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.488761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759aa8-0a97-4240-93d9-107d13fb228f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.488761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759aa8-0a97-4240-93d9-107d13fb228f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.488761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759aa8-0a97-4240-93d9-107d13fb228f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.492023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 85/168] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:45.493171 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6e9a4-9eaf-4897-abf8-4b5675b64c66 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.493458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6e9a4-9eaf-4897-abf8-4b5675b64c66 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.493733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6e9a4-9eaf-4897-abf8-4b5675b64c66 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.495172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 86/169] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:45.499321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6e9a4-9eaf-4897-abf8-4b5675b64c66 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.521739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8329-0c82-4608-8635-0040247a887c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.545398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.563299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.563929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.564208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.564486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] PATH_INFO: `/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.565602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] RBAC: Authorizing `identity:get_user(user_id=3a6f18cf4a5b4b16ab604dd0d0188854)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.567051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.568863 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/users/3a6f18cf4a5b4b16ab604dd0d0188854 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.573991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.586004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.611722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6e9a4-9eaf-4897-abf8-4b5675b64c66 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.649625 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.650271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.650552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.650826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.651557 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:45.651870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.653310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.669980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 87/171] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.669980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.686580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0857b0-5658-4679-b7da-7d4d84694869 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.721397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.722030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.722310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.722897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.723768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.725225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd0274-cca8-462a-8b6e-6abccc68aa7f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.727583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 85/172] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.733272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.733551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.733825 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.752585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:45.753772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:45.773524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607e3a8-9823-4856-bad0-b665d1649897 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.806864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.807517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.807794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.808061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.808559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.809406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:45.811000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff1ead-4a9b-467a-973f-cd665a7cd1cc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:45.812654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:45.813751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.814020 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.814290 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:45.820564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 86/174] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:45.820564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.840340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:45.841514 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:45.843502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:45.915676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 89/175] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:45.928076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:45.931480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b4a73-05f0-40d7-903e-b8e7e1854ed0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:45.967108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:45.967718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:45.968004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:45.968283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.027165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.038645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.040145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254b232-5ffa-4380-8f29-bb60e423ddee None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.076976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c9a89-3449-466b-8362-829bd15003e3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.118089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.118719 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.118996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.119300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] PATH_INFO: `/v3/users/1b9cf77b90164c2b900d5824c6606349` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.120414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9cf77b90164c2b900d5824c6606349)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.121831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.123630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 90/176] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/users/1b9cf77b90164c2b900d5824c6606349 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:46.128590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.137897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.147240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 87/177] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:46.148667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f05ef-87e1-4166-b754-379b28bb6b50 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.148967 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f05ef-87e1-4166-b754-379b28bb6b50 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.149245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f05ef-87e1-4166-b754-379b28bb6b50 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.150611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 88/178] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:46.155264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f05ef-87e1-4166-b754-379b28bb6b50 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.236269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220c8e-feb4-4c5e-ade6-f77a7db08a09 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.245221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.256168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f05ef-87e1-4166-b754-379b28bb6b50 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.269414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.270017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.270321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.270597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.271352 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:46.271661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.273122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.279044 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 91/179] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=800a18a269264faba7cbec11735ac79b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:46.279698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.308459 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.309767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.310060 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.310620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/14f88fa754624898b5c00bb859c3ef01/roles/6c219595326a412e8f909d29c07e8c6f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.313383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, user_id=14f88fa754624898b5c00bb859c3ef01, role_id=6c219595326a412e8f909d29c07e8c6f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.315011 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.329045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 89/180] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:46 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/users/14f88fa754624898b5c00bb859c3ef01/roles/6c219595326a412e8f909d29c07e8c6f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:46.344185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.361874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.406667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fdbb9-f95c-4cb7-a4c1-c8df3e3f1972 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.449706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.450498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.451124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.451124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.464645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:46.470801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:46.471950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:46.472865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 92/181] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:46.473517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.501908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbceeda-9770-4f88-be24-023b9f0984a1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.545120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.545975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.546400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.546667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.573482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.602987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f702d4-3693-4faa-a230-88b98b6dcdef None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.632553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.633112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.634028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.640078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.640717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.640996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.641270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.642429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.643900 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8a702-e834-4684-b9c6-436cfe006c7b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.650846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 93/182] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:46.650846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.650846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.650846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.666558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:46.667786 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:46.738062 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 94/183] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:46.739423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.764108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 90/184] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:46.765862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.820028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.844179 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4e4f1-ca2a-442e-8047-e60b7480b5ee None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.872554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:46.879877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28248f3-9d15-4395-b79a-0aa095156d2e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:46.881015 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.881755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.882143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.882485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.894719 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:46.900615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:46.901486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:46.902263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 95/185] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:46.902969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:46.927837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:46.929562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:46.930035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:46.930565 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:46.931532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:46.932004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:46.933748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:46.942504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 91/186] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:46.948220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.010879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51426b-c916-4c66-909b-965eeb4e034f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.050034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.050581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.050863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.051283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.052070 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=3a6f18cf4a5b4b16ab604dd0d0188854) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:47.052722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.054952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.071191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 96/187] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/role_assignments?user.id=3a6f18cf4a5b4b16ab604dd0d0188854&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.071423 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5bf44-404f-4f3b-8565-097d9aaee6e9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.071532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.088856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.120709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.121389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.122181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.122463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] PATH_INFO: `/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.124140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] RBAC: Authorizing `identity:get_project(project_id=f518bfe4dc6f4a00abda405cd6bd6960)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.126449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.129390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 92/188] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:46 2022] GET /identity/v3/projects/f518bfe4dc6f4a00abda405cd6bd6960 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.130274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.141569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.203602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdea4dd-5684-436f-af07-35b7b3551e14 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.244636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.245265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.245541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.245813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.246483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:47.246785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.248237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.258862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 97/189] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.260618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.289212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803fa5b-d8e9-4ac3-9ba5-db902e1b6167 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.345543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.346192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.346471 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.346755 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.362885 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.364742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.367313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 93/190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.368116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.393510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98b1f-b736-4b3a-845d-3354ce4ed7ae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.409211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.416801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.432681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.433447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.433863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.434156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.435311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.436756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.438536 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 98/191] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.439079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.483805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eafa2-ae3c-4192-8a4a-c2955cdcb00a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.519232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.519836 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.520118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.520392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.521123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=f518bfe4dc6f4a00abda405cd6bd6960, user.id=1b9cf77b90164c2b900d5824c6606349) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:47.521431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.523440 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.529272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 94/192] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/role_assignments?user.id=1b9cf77b90164c2b900d5824c6606349&scope.project.id=f518bfe4dc6f4a00abda405cd6bd6960&role.id=800a18a269264faba7cbec11735ac79b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.529894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.563863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4cf3b-0cd9-4c28-97e5-6be3722cfb3d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.613504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.615110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.615449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.615732 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.617583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.619747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.622394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 99/193] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:47.623015 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.650210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6926e6-957a-4a4f-b0ae-a067ba11d475 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.652249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.695125 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.695912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.696262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.696605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.712326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.781401 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae493-f02e-4f3d-85bf-dbe3956854ce None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:47.800293 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.801899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.818570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:47.819225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:47.819545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:47.819827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:47.820487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:47.821937 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:47.859180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 100/194] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:47 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:47.859180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.919112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 95/195] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:47.928583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:47.930847 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:47.991594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a05ef0-aa58-4b6f-8eae-07e9faac2595 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:48.001131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:48.039095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:48.040109 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.040588 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.041056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.070871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147cbf2-2bd0-4f83-af93-1b90442bf893 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:48.122728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:48.123405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.123714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.124284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.125699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:48.127339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:48.129745 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 96/196] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:48.132688 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:48.134211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:48.135720 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:48.227722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:48.269509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ab924-4e80-451b-a8b5-c11ceeaf665c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:48.275072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 101/197] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:48.280701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:48.289617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:48.319773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:48.320755 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.321397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.321871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.322822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=e4b2bf397f894013b8bb79b9a936d3ac) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:48.324188 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:48.325859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:48.333812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 97/198] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:48 2022] GET /identity/v3/role_assignments?user.id=e4b2bf397f894013b8bb79b9a936d3ac&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:48.427313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c546-7438-436f-9e90-55eed7142964 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:48.471775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:48.472838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.473352 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.473850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.475457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:48.477257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a13629-1018-4fff-8722-db85ccc8e8d7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:48.479359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 102/199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:48 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:48.485457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:48.605292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d7dc-2189-45e9-bf32-4daf1605f649 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:48.641679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:48.642698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.643288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.643768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.644698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=e4b2bf397f894013b8bb79b9a936d3ac) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:48.645213 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:48.646869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb2dca-e4c8-4c4f-a1d1-d3d0238ef07c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:48.649595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:48.654307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 98/200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:48 2022] GET /identity/v3/role_assignments?user.id=e4b2bf397f894013b8bb79b9a936d3ac&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:48.746948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68335-ec5a-49c7-b0c5-ab45e6a6b140 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.747640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68335-ec5a-49c7-b0c5-ab45e6a6b140 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.748136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68335-ec5a-49c7-b0c5-ab45e6a6b140 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.749859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 103/201] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:48.761797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:48.762429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:48.762907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:48.792447 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:48.794091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:48.893258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 99/202] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:49.296288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68335-ec5a-49c7-b0c5-ab45e6a6b140 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:49.396610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:49.424242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68335-ec5a-49c7-b0c5-ab45e6a6b140 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:49.464476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:49.465191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:49.465479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:49.465756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:49.476741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:49.490022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:49.490903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:49.491788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 104/203] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:49.497329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:49.609271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790f5aa-a681-4a35-861c-efa4d08a04b8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:49.642911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:49.657891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:49.658535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:49.658822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:49.659097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:49.659882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:49.660192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:49.661568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:49.674119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 100/204] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:49.680039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:49.784813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88156bd3-4fa5-4a58-b1c6-3be9750e3a09 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:49.821223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:49.821871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:49.822154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:49.822432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] PATH_INFO: `/v3/users/14f88fa754624898b5c00bb859c3ef01` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:49.823615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] RBAC: Authorizing `identity:get_user(user_id=14f88fa754624898b5c00bb859c3ef01)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:49.825103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:49.826882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 105/205] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:49 2022] GET /identity/v3/users/14f88fa754624898b5c00bb859c3ef01 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:49.832636 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:49.940442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1c9f-8ab9-45e3-8dcf-54ae1dceebe5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:49.975032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:49.975741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:49.976028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:49.976307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.046417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:50.047924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:50.049850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:50.153544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 101/206] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:50.159395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:50.163441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:50.263400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18618d9-912d-4a5c-8a97-d3dc828fb29c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:50.298097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:50.298738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.299016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.299464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.300607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:50.302065 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3801832-8da8-47d2-96a5-3828ea884314 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:50.304178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 106/207] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:50 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:50.310421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:50.418705 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e921-c191-4c66-b314-468cdf7150af None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:50.420438 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:50.455720 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:50.456341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.456625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.456903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.457643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30, user.id=14f88fa754624898b5c00bb859c3ef01) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:50.457956 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:50.459496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8674e5-af1b-43d5-96e1-71620709bbd2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:50.467659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 102/208] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:50 2022] GET /identity/v3/role_assignments?user.id=14f88fa754624898b5c00bb859c3ef01&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=6c219595326a412e8f909d29c07e8c6f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:50.487589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c86f-c894-4ba7-be64-dbfa00c3ac30 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.487891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c86f-c894-4ba7-be64-dbfa00c3ac30 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.488154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c86f-c894-4ba7-be64-dbfa00c3ac30 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.489512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 107/209] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:50.495871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ceefb-3dcb-47da-b315-9df6f57317e2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.496211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ceefb-3dcb-47da-b315-9df6f57317e2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.496496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ceefb-3dcb-47da-b315-9df6f57317e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.519246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-017ceefb-3dcb-47da-b315-9df6f57317e2 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:50.520409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-017ceefb-3dcb-47da-b315-9df6f57317e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:50.595922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 103/210] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:50.912388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1c18-6f39-49a7-a7a1-7e2b721752f8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.913612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1c18-6f39-49a7-a7a1-7e2b721752f8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.914125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1c18-6f39-49a7-a7a1-7e2b721752f8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.915708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 108/211] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:50.921149 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b08481-0b0a-4dac-aa62-902a3c5ce9aa None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:50.921645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b08481-0b0a-4dac-aa62-902a3c5ce9aa None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:50.922107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b08481-0b0a-4dac-aa62-902a3c5ce9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:50.928678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1c18-6f39-49a7-a7a1-7e2b721752f8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:50.948244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-97b08481-0b0a-4dac-aa62-902a3c5ce9aa None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:50.948709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-97b08481-0b0a-4dac-aa62-902a3c5ce9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:51.034447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.036534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 104/212] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:51.067781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1c18-6f39-49a7-a7a1-7e2b721752f8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.102951 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.103621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.103902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.104178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.106156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2d0e-130d-4859-979c-11029e54f300 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.106716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2d0e-130d-4859-979c-11029e54f300 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.107334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2d0e-130d-4859-979c-11029e54f300 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.109016 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 105/213] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:15:51.114285 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.114559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.114841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.115021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:51.120857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:51.121735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:51.122504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 109/214] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:15:51.127764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.143260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:51.144454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:51.155468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.214242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 106/215] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:51.254039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40490e92-e11e-439a-9f6a-9b33ed717bde None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.283125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.289208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.289829 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.290115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.290395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.291073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:51.291415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.292798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.301681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 110/216] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:51.308023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.345438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.424253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5a9d8-8a90-412d-ba41-14f990b9374f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.462191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.463278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.463856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.464377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] PATH_INFO: `/v3/users/fe415039461f4629a59ecc41f62bafd5` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.477391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e67f-f50c-44b9-babc-49eb43f5f0d3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.479526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=fe415039461f4629a59ecc41f62bafd5)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.481333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.483493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 107/217] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/users/fe415039461f4629a59ecc41f62bafd5 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:51.488521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.492043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.517067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.518108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.518395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.518674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.520159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.521873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.524918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 111/218] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:51.529650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.592802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.610151 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6dd7c-5cfa-44b9-a86a-f700787b2aa3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.637778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537c093-2ecc-4b06-b058-ad463f070953 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.646736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.647441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.647781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.648072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.673664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.674288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.674838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.675118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.685928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:51.690733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:51.691885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 112/219] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:15:51.692655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.716811 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.718206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6a40d-5301-40b2-bcdc-44da87f34eda None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.720030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.792775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15b2d-d7c1-4c54-b386-8e0bea8ae91f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.812151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.814759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 108/220] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 03:15:51.816040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30076-6340-4dca-b0a5-8dc1f45f1e99 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.816328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30076-6340-4dca-b0a5-8dc1f45f1e99 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.816603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30076-6340-4dca-b0a5-8dc1f45f1e99 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.817839 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 109/221] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:51.818535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d30076-6340-4dca-b0a5-8dc1f45f1e99 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.833693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.834301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.834579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.834876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.836083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.837522 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.857953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 113/222] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:51 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:15:51.859018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:51.936173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d30076-6340-4dca-b0a5-8dc1f45f1e99 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:51.937963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:51.975993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:51.976533 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.976813 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.977085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.977778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:51.978085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:51.979540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7beca3-0538-43ca-96b3-ac342a937d4a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:51.994006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 110/223] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:51.994006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:51.994006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:51.994006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:51.994461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460caab-3f39-4d61-b8e1-c257e240a832 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.010267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:52.011555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:52.030359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.031739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.032019 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.032290 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.033433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.034887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.036717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 114/224] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:51 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:52.037634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.086750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 111/225] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:52.095278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.095394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:52.161149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ea24b-d837-4029-bfc3-148592ba1204 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.203914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.204647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.204999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.205341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.206492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.207990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.209570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 115/226] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:52.215510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.221903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:52.235405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c199b-176b-4ccd-8893-ac911d852fc1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.274100 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.275120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.275666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.276136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.277127 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=fe415039461f4629a59ecc41f62bafd5) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:52.277733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.279591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.286280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 112/227] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/role_assignments?user.id=fe415039461f4629a59ecc41f62bafd5&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=800a18a269264faba7cbec11735ac79b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:52.333740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce9a7-7ffa-4ffe-9211-7c6684b7c0cd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.348581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:52.372316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.372981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.373260 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.373533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.374763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.376267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.378176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 116/228] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:52.384682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.425478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.503661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82aa44-0577-4543-8836-9a1ebae7592c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.542376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.543182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.543458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.543837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/e4b2bf397f894013b8bb79b9a936d3ac/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.545781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, user_id=e4b2bf397f894013b8bb79b9a936d3ac, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.547454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.549703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f1af-2c12-488a-982a-290263dce14e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.566096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 113/229] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:52 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/e4b2bf397f894013b8bb79b9a936d3ac/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:52.599896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.601198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.601541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.601818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.614443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:52.620412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:52.621337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:52.622750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 117/230] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:15:52.627932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.673743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:52.756855 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeca4b0-4f01-4b8d-a81a-1ea329e2cd11 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.807764 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.808380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.808663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.808935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.809601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:52.809911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.811451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.820418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 114/231] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:52.832293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:52.878401 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:52.944583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cb8aa-ded9-49e7-b297-38819224c8d1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:52.986930 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:52.987593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:52.987869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:52.988141 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:52.991837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:52.993289 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:52.995259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 118/232] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:53.000437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:53.102217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:53.130858 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2624b-37e6-4c33-9eaf-7e65ec4437fd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:53.170860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:53.171516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:53.171795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:53.172063 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:53.175281 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:53.176840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d0048-d9a3-444e-8a1f-b8becb5c4e12 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:53.178801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 115/233] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:52 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:53.184446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:53.313196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ada90-046e-4a83-ad2e-06485d7f49a3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:53.323970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:53.359177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:53.359796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:53.360082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:53.360355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/e4b2bf397f894013b8bb79b9a936d3ac/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:53.364912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, user_id=e4b2bf397f894013b8bb79b9a936d3ac, role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:53.366396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82289ec-3059-4373-8fb2-a248c1c21843 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:53.405421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 119/234] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:53 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/e4b2bf397f894013b8bb79b9a936d3ac/roles/4363620f6a3647d0a382928ffef15ec7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:54.616671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a9ff-bac6-446a-b5fd-d6e74313c355 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.617194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a9ff-bac6-446a-b5fd-d6e74313c355 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.617194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a9ff-bac6-446a-b5fd-d6e74313c355 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.618949 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 116/235] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:54.631562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a891414-3f9f-4471-b2cd-0adb48b0d1a7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.632600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a891414-3f9f-4471-b2cd-0adb48b0d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.632600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a891414-3f9f-4471-b2cd-0adb48b0d1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.660145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8a891414-3f9f-4471-b2cd-0adb48b0d1a7 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:54.661351 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8a891414-3f9f-4471-b2cd-0adb48b0d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:54.729612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 120/236] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:54.760130 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54d6e-a5aa-47e0-bc00-10d9b0c7c61f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.760448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54d6e-a5aa-47e0-bc00-10d9b0c7c61f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.760679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54d6e-a5aa-47e0-bc00-10d9b0c7c61f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.762275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 117/237] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:54.767860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ae-3ebe-460c-b41a-0169201e2ea4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.768149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ae-3ebe-460c-b41a-0169201e2ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.768424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ae-3ebe-460c-b41a-0169201e2ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.786397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fbfa23ae-3ebe-460c-b41a-0169201e2ea4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:54.787601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fbfa23ae-3ebe-460c-b41a-0169201e2ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:54.891055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 121/238] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:54.946969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb12de-3e08-42c7-9e4e-34073b111a78 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.947329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb12de-3e08-42c7-9e4e-34073b111a78 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.947557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb12de-3e08-42c7-9e4e-34073b111a78 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.949014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 118/239] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:54.954300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:54.954570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:54.954844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:54.973290 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:54.974450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:54.997719 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:55.056486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 122/240] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:55.103026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb12de-3e08-42c7-9e4e-34073b111a78 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.242070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb12de-3e08-42c7-9e4e-34073b111a78 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.266304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.284229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.285174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.285652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.286151 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.287482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:55.289315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:55.291405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 119/241] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:55.299212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.351570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:55.414436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af18fa8-fe88-4dbf-98e2-08ebaa61a2d8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.422678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31797d6a-422a-4620-8b3f-73f4e102ae43 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.449619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.450329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.450765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.451202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.451996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:55.453568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:55.460928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.461828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.462417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.462892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.475128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:55.479845 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 120/242] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:55 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:15:55.480725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:55.489170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:55.490314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:15:55.491369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:55.493334 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 123/243] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:15:55.499161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.527337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:55.637460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf24f7-2631-49fc-9a33-dd98f8f84710 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.680597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.681616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.682120 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.682653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.683602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:55.684122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:55.685744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef320a-51ea-40ca-b86c-d49d56780d1e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:55.688014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:55.698682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 124/244] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:55.700691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e777df-193b-4a52-a4fa-e43005126348 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.701217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e777df-193b-4a52-a4fa-e43005126348 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.701695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e777df-193b-4a52-a4fa-e43005126348 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.703383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 125/245] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:55.704403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e777df-193b-4a52-a4fa-e43005126348 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.731446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9673b-9f17-463d-9ff4-55a175bcd995 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.773297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.774291 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.774572 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.774843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.793985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-85988b85-589b-4ac2-848a-a1db6801427b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:55.801270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:55.802340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:55.804007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:55.806057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 121/246] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:55.807555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.807999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.808403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.808941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:55.830359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:55.831609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:55.837172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e777df-193b-4a52-a4fa-e43005126348 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:55.878323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:55.878967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:55.879276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:55.879554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] PATH_INFO: `/v3/users/829fe8090dc847769f89fb6516b1391a` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:55.894835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] RBAC: Authorizing `identity:get_user(user_id=829fe8090dc847769f89fb6516b1391a)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:55.896366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:55.898765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 126/247] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/users/829fe8090dc847769f89fb6516b1391a => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:55.899411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:55.914544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 122/248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:55.915582 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.003544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.057523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b67b-f739-4795-b19e-91e8e2a76668 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.069892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00bfa-07c6-4a06-9824-baecbab056ec None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.129624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.130253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.130537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.130814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.131510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:56.132350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.133719 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.138665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.139315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.139596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.139868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.152765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.162473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 127/249] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:56.175555 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.277023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.278522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f61da9-327e-42ab-a736-7461e5463e18 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.321983 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86a61-e472-4c6d-a4f2-0ea71176b044 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.380335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.386845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.391175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.391175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.391175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] PATH_INFO: `/v3/users/fe415039461f4629a59ecc41f62bafd5` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.391876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] RBAC: Authorizing `identity:get_user(user_id=fe415039461f4629a59ecc41f62bafd5)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.395162 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.395162 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 128/250] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/users/fe415039461f4629a59ecc41f62bafd5 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:56.395816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.431227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 123/251] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:56.432688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b225a-ce77-4b50-9c44-9e815c690048 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.432979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b225a-ce77-4b50-9c44-9e815c690048 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.433253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b225a-ce77-4b50-9c44-9e815c690048 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.434859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 124/252] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:56.435595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b225a-ce77-4b50-9c44-9e815c690048 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.485350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.523131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fc57-6c18-4689-8eeb-42025766826c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.550847 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b225a-ce77-4b50-9c44-9e815c690048 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.561367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.562017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.562302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.562578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.574571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:56.580434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:56.581326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:15:56.582121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 129/253] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:56.582770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.597629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.598963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.599737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.600211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.602342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.604798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb696089-2553-4722-aa8c-d1644c740b5b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.607325 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 125/254] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:56.608719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.609754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.610220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.634336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:56.635572 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:56.661334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.709541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 126/255] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:56.710547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.715360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93222-86bd-42fd-908a-6af3fc97cfe7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.750362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:56.770810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.771809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.772095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.773105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.776534 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.780748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.782904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 130/256] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:56.784653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:56.891032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bcb5f-f872-4988-878e-4f6d3e5a626d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.925573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15205f-305e-4600-9529-43c92a1b030e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:56.964797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.965478 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:56.966106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.966386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.966659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.967848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:56.968129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:56.968404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/fe415039461f4629a59ecc41f62bafd5/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:56.969386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:56.969701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.971625 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, user_id=fe415039461f4629a59ecc41f62bafd5, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:56.973050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.976301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:56.987544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 127/257] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:56.988404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.000023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 131/258] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:15:56 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/fe415039461f4629a59ecc41f62bafd5/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:15:57.000951 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.050174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.085524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.137735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934a720-def6-4761-b71a-7f02d2ca8e8d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:57.144691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebc7d-9552-47ae-a008-659af651d32d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:57.177892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:57.179676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:57.180117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:57.180537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:57.181610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=829fe8090dc847769f89fb6516b1391a) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:57.182437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:57.184647 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:57.190959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:57.192328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 128/259] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:56 2022] GET /identity/v3/role_assignments?user.id=829fe8090dc847769f89fb6516b1391a&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=6c219595326a412e8f909d29c07e8c6f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:57.193352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.196053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:57.196581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:57.197462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:57.201067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:57.204162 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:57.213063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 132/260] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:57.218489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.346442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.365040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a0a58-366f-42af-ab95-c0846d82cedf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:57.421191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:57.421772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:57.422057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:57.422334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:57.435323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:57.441569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:57.442697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:15:57.443687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:57.446140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 129/261] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:57.452002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.514021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ef5a5-f5a9-4e53-9e85-9b1a73a303c9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:57.556700 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.602140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:57.602768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:57.603045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:57.603346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:57.668502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57370-fac0-404d-bc0e-e11edfe1f934 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:57.733954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:57.735236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:57.735657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:57.736085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:57.736891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:57.737348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:57.738161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:57.739993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:57.741673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:57.758474 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 130/262] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:57.764503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.777829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.874783 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 133/263] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:15:57.882121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:57.926883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:57.956362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94019f-0924-47e8-8ab2-e20aeb063ea7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.027959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.028532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.028812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.029091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] PATH_INFO: `/v3/users/e4b2bf397f894013b8bb79b9a936d3ac` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.030280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b2bf397f894013b8bb79b9a936d3ac)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.031769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.033703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 131/264] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/users/e4b2bf397f894013b8bb79b9a936d3ac => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:58.043687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a21f99-78de-4abb-b727-a2cf1be9ce1a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.050328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:58.106557 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.107225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.107513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.107792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.108945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.110410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.112438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 134/265] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:57 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:58.118617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:58.184777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:58.231926 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d402ac-58f9-42da-854f-4b24f6489415 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.247425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:58.258090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9cf72-00e0-419d-9f9e-447c5aae8a7e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.293482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.294034 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.294478 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.294893 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.297147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.298016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.298450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.298867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.299798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=e4b2bf397f894013b8bb79b9a936d3ac) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:58.300258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.301881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.309329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 135/266] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:58 2022] GET /identity/v3/role_assignments?user.id=e4b2bf397f894013b8bb79b9a936d3ac&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:58.378525 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.380037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491c74e-daeb-4436-8c95-b1685bd04a2d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.476056 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:58.487836 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 132/267] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:15:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 03:15:58.496475 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:58.504207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a08-cac8-48c2-9b69-77ca7fb5db49 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.504503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a08-cac8-48c2-9b69-77ca7fb5db49 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.504753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a08-cac8-48c2-9b69-77ca7fb5db49 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.506226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 133/268] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:58.523179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.523179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.523179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.555252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:58.555252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:58.689140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0deb0-185d-4db7-a9c6-d804d9959cd9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.692160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 134/269] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:15:58.747972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.748678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.748918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.749192 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.750406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.751944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668e668-2b1e-448d-9500-c35f2acc88ea None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.754026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 136/270] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:15:58 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:58.759826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:58.897037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d606c2d-5d97-486b-9b02-67aa25cb9e58 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:58.909484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:58.934123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a1457-8777-4cfb-8723-2ba71a2f2d00 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.934427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a1457-8777-4cfb-8723-2ba71a2f2d00 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.934678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a1457-8777-4cfb-8723-2ba71a2f2d00 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.936068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 137/271] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:15:58.942104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.942400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.942676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.951898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:58.952804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:58.953286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:58.953759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:58.954691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=e4b2bf397f894013b8bb79b9a936d3ac) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:58.955257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:58.957876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:58.973139 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 135/272] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:15:58 2022] GET /identity/v3/role_assignments?user.id=e4b2bf397f894013b8bb79b9a936d3ac&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:58.976711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:15:58.977948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:15:58.997040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.017528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:59.088823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 138/273] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:15:59.168599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9d9fd-0ede-4c5c-ae03-0c8487b171cf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.217386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.218393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.218963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.219452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.235341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:15:59.240951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:15:59.241905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:15:59.242724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:15:59.245300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 136/274] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:15:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:15:59.250847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.368761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549ec42-4ecd-4625-9f65-e32fdb3232c4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.403590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.404210 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.404486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.404754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.405417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:59.405716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:59.407080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:59.416197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 139/275] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:15:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:59.422114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.455047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:59.476003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.511509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:59.531751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7567abe-88b7-4eb7-81e6-e2b99e3f2950 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.566771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.567432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.567880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.568317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] PATH_INFO: `/v3/users/582bc61b28e142f49b39d88095588bc4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.582207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] RBAC: Authorizing `identity:get_user(user_id=582bc61b28e142f49b39d88095588bc4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:59.583910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:59.586630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 137/276] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:15:59 2022] GET /identity/v3/users/582bc61b28e142f49b39d88095588bc4 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:15:59.593029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.601083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5e95a-d53f-4d5f-ae16-1c59ab7fd6b0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.641873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.643236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.643665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.644081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.646025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:59.648524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:59.651370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 140/277] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:15:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:59.656554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:15:59.698019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:59.711262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecffa-78bf-44e4-be1c-1d1e760e05d9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.747187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.748041 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.748465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.748877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.749749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, user.id=582bc61b28e142f49b39d88095588bc4, scope.system=all) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:15:59.750228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:59.751854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde67d90-2483-4f37-9072-c4d58ce41f9e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:59.757692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 138/278] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 03:15:59 2022] GET /identity/v3/role_assignments?user.id=582bc61b28e142f49b39d88095588bc4&scope.system=all&role.id=800a18a269264faba7cbec11735ac79b => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:15:59.759649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:15:59.774815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da60f-8921-4c25-b1a0-9a0a3c3b22e2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:15:59.819042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:15:59.819869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:15:59.820224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:15:59.820608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:15:59.821369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:15:59.822940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe982c-b3df-46a8-8ed1-7a50d37c207c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:15:59.851687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 141/279] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:15:59 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:16:00.110156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e370cf-c475-4dac-8324-4e5c89354ac4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.110680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e370cf-c475-4dac-8324-4e5c89354ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.110756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e370cf-c475-4dac-8324-4e5c89354ac4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.112132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 139/280] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:00.117582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.117872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.118148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.137366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175936-28df-4ed3-88f5-49d830a10f74 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.137697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175936-28df-4ed3-88f5-49d830a10f74 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.137923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175936-28df-4ed3-88f5-49d830a10f74 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.139030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:00.139511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 140/281] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:00.140268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:00.144780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.145067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.145342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.165105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:00.166311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:00.228084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 142/282] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:00.253111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 141/283] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:00.532664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:00.541891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:00.647410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:00.677436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:00.682023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310202-9e4e-4bf1-b2ba-9d9cac8110b6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:00.710914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ce74f-87a7-4112-90e3-99d1226ef8be None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:00.717810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:00.718720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.719255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.719755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.730929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:00.736371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:00.737236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:00.738002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:00.740267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 143/284] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:00.746324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:00.781061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:00.781761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.782047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.782322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.816451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:00.827547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:00.829327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:00.830905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 142/285] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:00.838326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:00.888376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fd7b4-199a-4a48-8a84-71ea721a5741 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:00.943589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:00.954964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:00.955613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:00.955899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:00.956172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:00.956841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:00.957148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:00.958521 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:00.970355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 144/286] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:00.978436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.057048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac90d-6603-46d1-a9d2-36484497ec83 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.126860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.127510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.127758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.128035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.128703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:01.129008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.130424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.150332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 143/287] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:01.155875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.160597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.178015 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be2e8-a34a-49ba-ad2c-6c63df9d9bfa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.244477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.245452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.245926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.246413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] PATH_INFO: `/v3/users/fe415039461f4629a59ecc41f62bafd5` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.251827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] RBAC: Authorizing `identity:get_user(user_id=fe415039461f4629a59ecc41f62bafd5)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.253506 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.255661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 145/288] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:00 2022] GET /identity/v3/users/fe415039461f4629a59ecc41f62bafd5 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:01.260929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.265338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de65-0a5f-4c3e-b7c9-fecee48b7d3a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.303135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.304610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.304891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.305218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] PATH_INFO: `/v3/users/829fe8090dc847769f89fb6516b1391a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.306388 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] RBAC: Authorizing `identity:get_user(user_id=829fe8090dc847769f89fb6516b1391a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.308191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.309920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 144/289] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:01 2022] GET /identity/v3/users/829fe8090dc847769f89fb6516b1391a => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:01.319170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.372383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.439497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.439978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b405ef8-570e-48a0-9b05-06e3cf61c502 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.496238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5726e6-cae6-4775-8d2e-0430e6a48be5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.496967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.497813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.498277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.498735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.561091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.561964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.562687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.563196 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.564547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.566228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.568579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 145/290] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:01 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:01.575879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.576321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.577227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.639537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.681278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 146/291] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 03:16:01.686821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.773518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a417-afd8-4656-b305-e70131ac2433 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.791208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e391-94a4-4078-8959-f80d00d9e1dd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.826668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.827703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.828136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.828434 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.828887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.828990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/829fe8090dc847769f89fb6516b1391a/roles/6c219595326a412e8f909d29c07e8c6f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.829237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.829582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.830970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.831679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, user_id=829fe8090dc847769f89fb6516b1391a, role_id=6c219595326a412e8f909d29c07e8c6f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.832583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.833544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdc182-cd6c-4acf-9eb3-2b661da38e1d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:01.843830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 147/292] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:01 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:01.843830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:01.850637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.857137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 146/293] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:01 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/users/829fe8090dc847769f89fb6516b1391a/roles/6c219595326a412e8f909d29c07e8c6f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:01.873902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:01.954737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a308b9-cb01-4021-a541-09f8c4e21baa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:01.993390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:01.994213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:01.994676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:01.995137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:01.996325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=fe415039461f4629a59ecc41f62bafd5) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:01.996641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:01.998070 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5028c7-e597-4174-9ac3-cf9e8bc08ab3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:02.008021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 148/294] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:01 2022] GET /identity/v3/role_assignments?user.id=fe415039461f4629a59ecc41f62bafd5&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=800a18a269264faba7cbec11735ac79b => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:02.338496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b2645-b821-48b9-98e5-b4153680f0f2 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.339530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b2645-b821-48b9-98e5-b4153680f0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.340038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b2645-b821-48b9-98e5-b4153680f0f2 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.341873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 147/295] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:02.347437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ef45-2ce2-4b4d-969d-39e2cd0438f6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.347728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ef45-2ce2-4b4d-969d-39e2cd0438f6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.348005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ef45-2ce2-4b4d-969d-39e2cd0438f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.370815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4d53ef45-2ce2-4b4d-969d-39e2cd0438f6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:02.372034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4d53ef45-2ce2-4b4d-969d-39e2cd0438f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:02.456058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 149/296] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:02.701387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ce42d-9d89-4741-99e6-4647893fe2f7 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.701574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ce42d-9d89-4741-99e6-4647893fe2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.701854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ce42d-9d89-4741-99e6-4647893fe2f7 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.703455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 148/297] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:02.708609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c9cb-4bb8-48c5-bdf9-87963cd8a47e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.708893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c9cb-4bb8-48c5-bdf9-87963cd8a47e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.709167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c9cb-4bb8-48c5-bdf9-87963cd8a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.726881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ce42d-9d89-4741-99e6-4647893fe2f7 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:02.727995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3e12c9cb-4bb8-48c5-bdf9-87963cd8a47e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:02.729311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3e12c9cb-4bb8-48c5-bdf9-87963cd8a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:02.747747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:02.800907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 150/298] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:02.802420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4453fb-0434-4ff0-8904-b1ad4edc6293 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.802850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4453fb-0434-4ff0-8904-b1ad4edc6293 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.803727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4453fb-0434-4ff0-8904-b1ad4edc6293 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.806388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 151/299] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:02.811242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.811682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.812118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.831027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:02.832558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:02.845825 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ce42d-9d89-4741-99e6-4647893fe2f7 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:02.883004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:02.883842 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:02.884216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:02.884780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:02.902094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:02.911873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:02.912928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:02.913711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:02.915119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 152/300] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:02.919264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 149/301] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:02.925673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.014777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.033696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c118d-e89d-440d-bce4-a071724b1571 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.068014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.068677 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.069518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.069790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.070443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:03.070741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.072201 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.080616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 153/302] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:03.085861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.132685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.146914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.192616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9eb6b-0214-4663-a2d5-bf25a0943c34 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.235248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.235885 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.236171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.236443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] PATH_INFO: `/v3/users/582bc61b28e142f49b39d88095588bc4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.237564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] RBAC: Authorizing `identity:get_user(user_id=582bc61b28e142f49b39d88095588bc4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.239043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.247182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 150/303] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/users/582bc61b28e142f49b39d88095588bc4 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:03.247182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.264408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3742-88b5-4f1a-ba78-58009e246e9c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.302170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.302761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.302991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.303576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.314552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:03.319954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:03.320839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:03.321767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 154/304] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:03.326404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.330773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.359965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421578ca-a123-42be-bb5f-1dbf402a222d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.391631 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.401029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.401860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.401966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.402183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.403331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.404849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.406445 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 151/305] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:03.408024 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.438371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c5d-90fb-4559-aa8e-5f786482a405 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.474901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.476180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.476460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.476743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] PATH_INFO: `/v3/system/users/582bc61b28e142f49b39d88095588bc4/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.478124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=582bc61b28e142f49b39d88095588bc4, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.479510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.490543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 155/306] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 4 03:16:03 2022] PUT /identity/v3/system/users/582bc61b28e142f49b39d88095588bc4/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:03.496097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.533378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb3af-9f7e-40b2-8be8-dbdbf71a4e6c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.575686 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.576472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.577238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.577615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.578026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.578810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:03.579272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.581012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.594931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 152/307] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:03.600279 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.626655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ba97f-36f7-4b5e-9574-e3643f90f14c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.671970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.673051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.673051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.673234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] PATH_INFO: `/v3/groups` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.673767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.675244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.691287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.694477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 156/308] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 4 03:16:03 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:03.751824 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd4705-715c-45c8-b42f-e0bd30d51ed1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.795421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.796461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.796944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.797446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] PATH_INFO: `/v3/users/38fb568d5efe43a2ae8022700663c20b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.813997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] RBAC: Authorizing `identity:get_user(user_id=38fb568d5efe43a2ae8022700663c20b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.815996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81ebaf-cc07-4dac-b0ea-6970ca12bfc2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.818307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 153/309] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/users/38fb568d5efe43a2ae8022700663c20b => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:03.824348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:03.940747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce090-8ed6-44ac-afda-a760a00b05a0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:03.967990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:03.976784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:03.977413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:03.977695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:03.977972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:03.978708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, user.id=38fb568d5efe43a2ae8022700663c20b, scope.system=all) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:03.979020 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:03.980505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2ae5-f05c-4b13-ad23-497db4704a42 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:03.986113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 157/310] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 03:16:03 2022] GET /identity/v3/role_assignments?user.id=38fb568d5efe43a2ae8022700663c20b&scope.system=all&role.id=6c219595326a412e8f909d29c07e8c6f => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:04.254733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef85a4-c5f3-4d45-aa0d-19c3e52ed9ee None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:04.254919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef85a4-c5f3-4d45-aa0d-19c3e52ed9ee None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:04.255225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef85a4-c5f3-4d45-aa0d-19c3e52ed9ee None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:04.256935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 154/311] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:04.262446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:04.262902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:04.262996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:04.282613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:04.283852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:04.353962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 158/312] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:04.623934 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef85a4-c5f3-4d45-aa0d-19c3e52ed9ee None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:04.732353 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef85a4-c5f3-4d45-aa0d-19c3e52ed9ee None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:04.749305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:04.767172 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:04.767815 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:04.768092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:04.768362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:04.779307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:04.784129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:04.785006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:04.785997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 155/313] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:04.791161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:04.894173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287036f6-ca8e-449a-8260-a8c26cb73155 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:04.917317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:04.929802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:04.930422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:04.930700 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:04.930975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:04.931671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:04.931978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:04.933347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:04.941730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 159/314] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:04.947578 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:05.051454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fc075-16ff-4b4d-a074-327ee1e7a4b2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:05.087401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:05.088294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.088771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.089214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] PATH_INFO: `/v3/users/829fe8090dc847769f89fb6516b1391a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.090511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] RBAC: Authorizing `identity:get_user(user_id=829fe8090dc847769f89fb6516b1391a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:05.092158 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:05.094025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 156/315] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:04 2022] GET /identity/v3/users/829fe8090dc847769f89fb6516b1391a => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:05.099788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:05.202820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a79c9-b063-4cd6-ae1f-6707d0070181 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:05.238703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:05.239610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.240063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.240535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.295348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:05.312756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:05.314497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:05.416231 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 160/316] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:05.421550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:05.474819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:05.529827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1613-179f-4a8a-84e8-111743c1c14b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:05.568301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:05.569503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.569766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.570037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.571201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:05.572669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bfb4b-5857-46cc-9d63-25f691eec0c3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:05.574599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 157/317] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:05 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:05.580291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:05.664795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:05.683294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f32a-83d3-4787-9930-0a9d6ef968c7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:05.720239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:05.720875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.721154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.721429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.722187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d, user.id=829fe8090dc847769f89fb6516b1391a) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:05.722500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:05.724082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddf394-42e9-4c36-8c05-336d74a8fc17 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:05.731596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 161/318] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:05 2022] GET /identity/v3/role_assignments?user.id=829fe8090dc847769f89fb6516b1391a&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=6c219595326a412e8f909d29c07e8c6f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:05.836524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f39b-2d48-4a51-8316-15196ae879fe None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.837039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f39b-2d48-4a51-8316-15196ae879fe None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.837111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f39b-2d48-4a51-8316-15196ae879fe None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.838412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 158/319] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:05.844172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978663-e296-4303-98c8-a4915bf6e7ea None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:05.844455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978663-e296-4303-98c8-a4915bf6e7ea None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:05.844730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978663-e296-4303-98c8-a4915bf6e7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:05.863946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e0978663-e296-4303-98c8-a4915bf6e7ea None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:05.865180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e0978663-e296-4303-98c8-a4915bf6e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:05.946317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 162/320] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:06.019338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a93e7-70e5-42dc-b24a-96b1dfb46a71 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.019629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a93e7-70e5-42dc-b24a-96b1dfb46a71 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.019817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a93e7-70e5-42dc-b24a-96b1dfb46a71 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.021278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 159/321] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:06.026399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476393-d358-4e8f-8673-faf188ae20d8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.026692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476393-d358-4e8f-8673-faf188ae20d8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.026962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476393-d358-4e8f-8673-faf188ae20d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.045492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e8476393-d358-4e8f-8673-faf188ae20d8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:06.046658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e8476393-d358-4e8f-8673-faf188ae20d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:06.117264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 163/322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:06.211430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a93e7-70e5-42dc-b24a-96b1dfb46a71 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.221779 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ee87-0ecd-4903-a618-5b5b922c79fe None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.222069 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ee87-0ecd-4903-a618-5b5b922c79fe None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.222340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ee87-0ecd-4903-a618-5b5b922c79fe None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.223848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 164/323] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:06.229146 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.229429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.229699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.247420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:06.249661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:06.251924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.320131 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a93e7-70e5-42dc-b24a-96b1dfb46a71 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.324125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 165/324] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:06.333687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.355620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.356248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.356529 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.356803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.367537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:06.374176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:06.375696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:06.376492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 160/325] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:06.382881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.387530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.486053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f12ea-25fc-44cb-a1b6-4fcd4f19a0e9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.491780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d8144-cecf-47b9-9071-62f0cc51f5ac None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.520325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.520930 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.521213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.521488 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.522684 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:06.522991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:06.524385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:06.527647 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.528472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.528923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.529355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.530105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.531397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:06.533149 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:06.534866 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 161/326] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:06.537845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 166/327] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:06.538784 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.540294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.583827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.640886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d7eae-dbc2-438f-8df3-7ef486ad5dc6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.645553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d29748-3853-4532-8901-0e6bf162a4ac None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.676114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.676734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.677012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.677282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] PATH_INFO: `/v3/users/582bc61b28e142f49b39d88095588bc4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.678512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.678679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] RBAC: Authorizing `identity:get_user(user_id=582bc61b28e142f49b39d88095588bc4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:06.679114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.679426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.679693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] PATH_INFO: `/v3/groups` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.680147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:06.680308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:06.681703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 162/328] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/users/582bc61b28e142f49b39d88095588bc4 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:06.686493 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:06.686586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.698775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 167/329] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 4 03:16:06 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:06.704231 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.771434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.793515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3323efe-bba4-4ff0-93c0-8a4db18590d8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.804106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:06.824151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c13f4-1fb9-4c87-bd5c-0efae406e0b3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:06.833816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.835259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.835488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.835761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.847229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:06.852364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:06.853251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:06.854025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 163/330] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:06.859829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:06.866393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:06.867302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:06.867579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:06.867852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:06.868593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] RBAC: Adding query filter params (role.id=800a18a269264faba7cbec11735ac79b, user.id=582bc61b28e142f49b39d88095588bc4, scope.system=all) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:06.868905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:06.870338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:06.886427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 168/331] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/role_assignments?user.id=582bc61b28e142f49b39d88095588bc4&scope.system=all&role.id=800a18a269264faba7cbec11735ac79b => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:06.978198 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38fa8-33ca-4815-947b-154474452fe8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:07.016649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:07.017287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:07.017565 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:07.017841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:07.018546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:07.018848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:07.020334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:07.030422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 164/332] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:07.036255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:07.145323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721746-03a4-4783-a862-a95641dfb2c8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:07.166312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:07.185464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:07.186103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:07.186384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:07.186657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] PATH_INFO: `/v3/users/38fb568d5efe43a2ae8022700663c20b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:07.187865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] RBAC: Authorizing `identity:get_user(user_id=38fb568d5efe43a2ae8022700663c20b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:07.189303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcd462-5e4c-47ad-b6fc-26be45236886 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:07.191044 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 169/333] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:07 2022] GET /identity/v3/users/38fb568d5efe43a2ae8022700663c20b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:07.196773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:07.230778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:07.308750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51a84-6561-4aff-b2c3-76e3ef56745a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:07.347810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:07.348482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:07.348765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:07.349044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] PATH_INFO: `/v3/system/users/38fb568d5efe43a2ae8022700663c20b/roles/6c219595326a412e8f909d29c07e8c6f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:07.351252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=38fb568d5efe43a2ae8022700663c20b, role_id=6c219595326a412e8f909d29c07e8c6f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:07.352656 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b87afa-4f82-4c73-9cfc-6daa46833006 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:07.368337 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 165/334] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 4 03:16:07 2022] PUT /identity/v3/system/users/38fb568d5efe43a2ae8022700663c20b/roles/6c219595326a412e8f909d29c07e8c6f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:09.431832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186512a-2eed-495c-80e1-d8dce5688d88 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:09.432346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186512a-2eed-495c-80e1-d8dce5688d88 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:09.432590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186512a-2eed-495c-80e1-d8dce5688d88 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:09.434273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 170/335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:09.440594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8cdb3-5837-4009-b630-a559e2abe817 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:09.441007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8cdb3-5837-4009-b630-a559e2abe817 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:09.441148 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8cdb3-5837-4009-b630-a559e2abe817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:09.472734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b9b8cdb3-5837-4009-b630-a559e2abe817 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:09.474155 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9b8cdb3-5837-4009-b630-a559e2abe817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:09.554880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 166/336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:10.057026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186512a-2eed-495c-80e1-d8dce5688d88 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:10.067751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f94732-84c4-4358-b188-34026b57709f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.068278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f94732-84c4-4358-b188-34026b57709f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.068766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f94732-84c4-4358-b188-34026b57709f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.070265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 167/337] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:10.089223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.089778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.090256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.124201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:10.125707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:10.159443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:10.218442 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186512a-2eed-495c-80e1-d8dce5688d88 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:10.245887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 168/338] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:10.277295 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:10.278018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.278307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.278586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.294570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7b06d01b-bb0f-4f4d-bfdb-686aecd1ca5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:10.300472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:10.301413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:10.302202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:10.304461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 171/339] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:10.311163 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:10.450047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4508c-c53d-41da-9e8d-ff3753db5f7f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.450981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4508c-c53d-41da-9e8d-ff3753db5f7f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.451495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4508c-c53d-41da-9e8d-ff3753db5f7f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.453295 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 172/340] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:10.458860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.459517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.460008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.483869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:10.485379 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:10.488208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345d500-b814-4ab5-b123-c83e99abd4a4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:10.540508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:10.540806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:10.541620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.542038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.542458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.543302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:10.543618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:10.545012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c084867-75b6-4518-8a43-6bbcf4390d71 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:10.558114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 169/341] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:10.559401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bc37-d404-4094-85eb-37ff11b53a69 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.559852 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bc37-d404-4094-85eb-37ff11b53a69 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.560307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bc37-d404-4094-85eb-37ff11b53a69 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.561965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 170/342] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:10.567269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.567723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.568140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.574099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 173/343] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:10.575208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:10.596814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:10.598231 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:10.623171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:10.680797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea5f5d-1d1e-4205-a6ee-10e04fc58269 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:10.717508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:10.718126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:10.718407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:10.718678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:10.744807 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 171/344] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:10.767610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:10.790032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:10.791314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:10.791740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:10.904259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 174/345] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:10.910960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.018487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c4b2-a32a-495e-8365-557e1eacc2a3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.031491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.062872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922bc6c-637e-4076-9f7b-52ff6e71b9ba None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.066616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.067490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.067909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.068322 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.069741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.071357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.074421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 175/346] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:10 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:11.075360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.111502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.112747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.113224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.113763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.126881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6e09adc6-eb5f-40ce-92fb-fd3f92ffc26b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:11.132922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:11.133818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:11.134603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:11.137360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 172/347] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:11.138922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cab95-de8e-4427-b36b-04637eb12887 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.139476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cab95-de8e-4427-b36b-04637eb12887 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.139962 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cab95-de8e-4427-b36b-04637eb12887 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.141985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 173/348] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:11.142883 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cab95-de8e-4427-b36b-04637eb12887 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.157499 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.204264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28667d8-778e-4945-ab94-40980de456ea None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.249412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.250058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.250344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.250620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.265235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:11.271399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:11.272660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:11.273634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:11.276508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 176/349] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:11.278975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cab95-de8e-4427-b36b-04637eb12887 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.279243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.284400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.326599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.330818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.330818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.330818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.339273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.341106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98545-8430-4a5c-8373-7d968f97d2fa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.347414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 174/350] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:11.347414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.347414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.347414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.367566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:11.370906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:11.410736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4c892-b428-4f27-9524-5469188cf66e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.456816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.457422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.457706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.457980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.458651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:11.458957 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.461720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.462953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.466309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 175/351] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:11.468208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.481574 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 177/352] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:11.482456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.564832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.594357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3211e41-eef0-46c0-8a78-fd08ef768f4e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.610481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6279-36d7-4ca0-830d-68773b03901b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.632456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.633321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.633849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.634286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.645780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:11.649055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.649674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.649955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.650230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.650902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:11.651240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:11.652584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:11.653625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:11.654618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:11.655372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.657110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 176/353] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:11.658028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.664164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 178/354] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:11.664310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.733240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.781540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc521e-73eb-4f70-8aa6-7fcfb552d935 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.786391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed47324-0ca4-47bf-8c61-0b6a45d1c1c9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:11.813368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:11.823455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.824508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.824508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.824693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.827740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:11.828398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.828687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.831644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.831644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:11.831644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.831644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659709ce-b3fa-44d9-b356-0318ab0db5f5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.838516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 177/355] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=800a18a269264faba7cbec11735ac79b => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:11.839752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3c90-856d-4d54-8292-4935f5673839 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:11.840109 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3c90-856d-4d54-8292-4935f5673839 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:11.840390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3c90-856d-4d54-8292-4935f5673839 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:11.841738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 178/356] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:11.842403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a3c90-856d-4d54-8292-4935f5673839 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:11.923586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:11.925429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:11.984190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a3c90-856d-4d54-8292-4935f5673839 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.014868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.030576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.031219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.031504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.031779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.032464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:12.032775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:12.034162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:12.045660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 179/357] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:12.046586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.078658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 179/358] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:12.080039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.114051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.170430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d14309-7d0d-4b44-a2ba-dba46883e52e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.214238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.214873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.215173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.215459 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.220159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef590cac-b30a-492d-baa3-a3842076ff7e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.266202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.266881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.267211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.267464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.280338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-afd506d9-58b0-43d5-bf54-fc7edb4e488d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:12.285696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:12.286636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:12.287620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 180/359] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:12.288307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.288371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.289216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.296607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.311863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:12.313167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:12.315341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:12.317323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:12.422957 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 181/360] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:12.427546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.462867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.464430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 180/361] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:12.468829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.573159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1bc2d-093e-4097-ace1-db66cb097a6f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.584905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.607031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f72f1-0c06-43d8-942a-54f25724de27 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.623039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.623714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.623998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.624275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.651944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.652588 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.652870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.653144 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.654394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:12.656002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:12.658033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 181/362] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:12.658887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.710949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:12.712482 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:12.742569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.818408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1794-e65b-405d-b59f-c0b9e7768074 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.846212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 182/363] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:12.847480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.859102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:12.859748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:12.860038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:12.860309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:12.860982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:12.861288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:12.862671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:12.871194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 182/364] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:12.871980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:12.892675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:12.977527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ad318-4544-4265-958c-6fa1ecb60c3f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:12.996006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.006301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06a29a-08f6-4c3f-acb3-73541ef16042 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.019354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.020834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.020834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.021017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.023687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.025139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.026950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 183/365] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.027593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.064153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.064153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.064481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.064674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.067634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.069253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.070848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 183/366] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:12 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.073725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.159372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3529e-7a02-44f1-add7-4b769dce1112 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.181672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.202329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714237-54f2-46b1-9a3f-d3f3e63b6964 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.216701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.217708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.217994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.218268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.236580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:13.242259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:13.243274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 184/367] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:13.244410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.261659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.262023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.262306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.262585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.263910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.265364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.267214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 184/368] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.267787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.312864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.370306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac90e-7463-4577-ab86-c78d23230263 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.384617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a1551-52e6-4247-97f2-c5165baa51f5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.410226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.410855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.411132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.411429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] PATH_INFO: `/v3/users/38fb568d5efe43a2ae8022700663c20b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.413174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] RBAC: Authorizing `identity:get_user(user_id=38fb568d5efe43a2ae8022700663c20b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.414611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.420775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 185/369] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/users/38fb568d5efe43a2ae8022700663c20b => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.420775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.425961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.426699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.427053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.427430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.428949 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.430502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.432324 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 185/370] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.433027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.456841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.522947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362476-c4fa-4312-a4cc-5b800078b1ae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.537272 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.542840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b53e1-2223-482e-89e3-102047b577d1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.559261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.559871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.560155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.560430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.561160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:13.561532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.563010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.568450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 186/371] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 260 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.569711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.595270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.596528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.596990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.597644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.598970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:13.599658 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.602138 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.614704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 186/372] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.617343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.700368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f7de-d6d6-48dd-8fd6-4650aadbf51b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.717154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.740393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91852-243e-4017-bba3-def16bf4ce77 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.752720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.753366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.753649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.753922 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.755387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.757545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.759548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 187/373] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.760667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.787769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.788754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.789045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.789320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.790350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] RBAC: Adding query filter params (role.id=6c219595326a412e8f909d29c07e8c6f, user.id=38fb568d5efe43a2ae8022700663c20b, scope.system=all) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:13.790659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:13.792651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:13.811233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 187/374] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/role_assignments?user.id=38fb568d5efe43a2ae8022700663c20b&scope.system=all&role.id=6c219595326a412e8f909d29c07e8c6f => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:13.812233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:13.826152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:13.919248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07f8f-1553-4e80-ba36-f96ad633889c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:13.992702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:13.993736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:13.994234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:13.994711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:13.996652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:13.997487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.000597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.011075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406520-d410-42c5-b04c-5a6c1cf5b54e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.013569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 188/375] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:14.016168 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:14.097957 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:14.101152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.102618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.103230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.125833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:14.193361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad2a18-e7e9-4666-8b4d-a4cc1fd7df2c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.201241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.203063 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.211546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:14.237350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:14.238122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.238774 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.239224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.241291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:14.241722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.243424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6d72d-1e36-4fed-b827-6334fe1c1ea0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.251235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 189/376] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:14 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=800a18a269264faba7cbec11735ac79b => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:14.252835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3aa68-ce46-4506-8cd9-32e006441fd0 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.253255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3aa68-ce46-4506-8cd9-32e006441fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.253660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3aa68-ce46-4506-8cd9-32e006441fd0 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.255594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 190/377] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:14.262607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.265149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.266259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.298724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:14.300825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:14.396668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 188/378] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:14.401415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 191/379] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:14.402463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:14.478147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:14.547126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65248-9be4-46dd-8278-3d06b4f8b283 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.589518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:14.590154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.590429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.590703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] PATH_INFO: `/v3/projects/0c792b1b39084f70bab95471acabbdd7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.595846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] RBAC: Authorizing `identity:get_project(project_id=0c792b1b39084f70bab95471acabbdd7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.597296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.599316 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 189/380] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:14 2022] GET /identity/v3/projects/0c792b1b39084f70bab95471acabbdd7 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:14.605321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:14.733008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fabd7-7926-4bb8-9070-d9a35cd3d48d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.741072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:14.781227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:14.783417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:14.784423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.784855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.786138 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] PATH_INFO: `/v3/groups/e5645567b5624d7dbf905edbf58098bc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.794659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] RBAC: Authorizing `identity:get_group(group_id=e5645567b5624d7dbf905edbf58098bc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.796761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.800931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 192/381] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:14 2022] GET /identity/v3/groups/e5645567b5624d7dbf905edbf58098bc => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:14.806192 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:14.941981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408972-6bd0-4055-b18c-7aac3d25b87f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.975028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ead329-908d-4def-8d99-06289e4fda2a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:14.979090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:14.979798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:14.980198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:14.980522 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:14.981262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] RBAC: Adding query filter params (group.id=e5645567b5624d7dbf905edbf58098bc, role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=0c792b1b39084f70bab95471acabbdd7) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:14.981575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:14.983006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:14.989251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 193/382] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:14 2022] GET /identity/v3/role_assignments?group.id=e5645567b5624d7dbf905edbf58098bc&scope.project.id=0c792b1b39084f70bab95471acabbdd7&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:14.993319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:15.043493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:15.044139 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:15.044417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:15.044687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:15.063825 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:15.069361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:15.070199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:15.070993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 190/383] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:15.076713 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:15.127196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:15.242287 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e80b-d95a-4e9b-96ab-e1f58b45fdd4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:15.279473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:15.280143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:15.280430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:15.280709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:15.281375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:15.281687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:15.283048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:15.296142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 194/384] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:15.297764 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:15.423534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a83fb-a0c6-403a-9c46-d2680c3a6cc9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:15.471254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:15.471899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:15.472182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:15.472460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:15.476196 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:15.477653 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:15.479410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 191/385] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:15 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:15.485271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:15.586546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:15.593324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f3ae4-3ea3-467c-a330-ab912984377f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:15.629048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:15.629698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:15.629958 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:15.630229 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:15.631473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:15.632928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a3977-b0dd-49c4-92ec-6cc784bb0e9c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:15.634980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 195/386] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:15 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:15.640317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:15.691114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:15.776077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee809-0aec-4e31-afd9-8f9cee43b5c5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:15.826472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:15.827256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:15.827444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:15.827726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/groups/5418e8fe9f344091ae36cb24a376db36/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:15.832449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, group_id=5418e8fe9f344091ae36cb24a376db36, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:15.833903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5b5f9-442a-4e3a-bb92-62fd1dc3b44d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:15.852222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 192/387] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 03:16:15 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/groups/5418e8fe9f344091ae36cb24a376db36/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:16.356408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f57fe-dfb6-4eda-a937-943c20e7d97a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:16.356780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f57fe-dfb6-4eda-a937-943c20e7d97a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:16.356997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f57fe-dfb6-4eda-a937-943c20e7d97a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:16.358533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 196/388] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:16.366635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92729-a6cf-40ea-bfec-c0178e3cc7cf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:16.366923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92729-a6cf-40ea-bfec-c0178e3cc7cf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:16.367239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92729-a6cf-40ea-bfec-c0178e3cc7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:16.391384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f1e92729-a6cf-40ea-bfec-c0178e3cc7cf None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:16.392594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f1e92729-a6cf-40ea-bfec-c0178e3cc7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:16.480339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 193/389] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:16.848859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89c4aa-a83d-4839-8141-0c5340dce371 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:16.849368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89c4aa-a83d-4839-8141-0c5340dce371 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:16.849469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89c4aa-a83d-4839-8141-0c5340dce371 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:16.851048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 197/390] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:16.856314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991a5c8-e3ab-41ed-b3ed-37fd4ad0ecd1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:16.857119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991a5c8-e3ab-41ed-b3ed-37fd4ad0ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:16.857603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991a5c8-e3ab-41ed-b3ed-37fd4ad0ecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:16.876338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c991a5c8-e3ab-41ed-b3ed-37fd4ad0ecd1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:16.877804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c991a5c8-e3ab-41ed-b3ed-37fd4ad0ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:16.899218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:16.904613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89c4aa-a83d-4839-8141-0c5340dce371 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:16.976877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 194/391] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:17.028259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771babb-7853-4265-9462-aa328e07f221 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.028772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771babb-7853-4265-9462-aa328e07f221 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.029190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771babb-7853-4265-9462-aa328e07f221 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.030688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 195/392] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:16:17.043527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.044005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.044288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.068969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89c4aa-a83d-4839-8141-0c5340dce371 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.071354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:17.072528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:17.108613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.109267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.109549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.109822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.120547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:17.125821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:17.126685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:17.127819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 198/393] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:17.135230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.177971 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:17.226566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 196/394] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:17.290364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.306046 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120d86-1c21-43b5-9645-aaf82c9157b5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.351809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.352629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.352907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.353391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.354053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:17.354355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:17.355746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:17.375405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 199/395] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:17.381470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.453952 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:17.481590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ebda-26b2-486f-873d-ab041cb92f47 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.534803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:17.545953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.546614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.546890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.547175 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.569102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cf0c-6ea8-4fcd-b455-909107657d24 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.571461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:17.577052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:17.577938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:17.579116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 197/396] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:17.579819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.608189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.608882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.609157 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.609427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.610591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:17.612058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:17.619166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 200/397] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:17.619166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.697999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f5dce-b217-44c4-b1af-e29e63a361c6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.717033 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733f09f-48fe-4964-b411-d763cf152782 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.741479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.742530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.743410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.744141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.750098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.750702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.750984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.751279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.751934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:17.752235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:17.753591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:17.758089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:17.761845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 201/398] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:17.762617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5d7ecc6d-90a6-4da9-a709-d45a46f01b50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:17.764524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:17.765393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:17.766169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 198/399] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:17.767794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47689a-e667-40e6-b0f5-4084303a2c32 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.768136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47689a-e667-40e6-b0f5-4084303a2c32 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.768468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47689a-e667-40e6-b0f5-4084303a2c32 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.771953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 199/400] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:17.772802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47689a-e667-40e6-b0f5-4084303a2c32 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.782661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:17.785023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:17.874779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47689a-e667-40e6-b0f5-4084303a2c32 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.877614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463200f6-7845-4dda-a57b-794d39d6db99 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:17.908990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.909595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.909874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.910142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.910797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:17.911096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:17.912985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:17.915655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:17.916474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.916939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.917391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.918674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:17.920309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8acb48-6400-4e04-980e-3683feb6a904 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:17.921024 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 200/401] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:17.921801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:17.922471 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 202/402] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:17.923764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:17.924204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:17.924643 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:17.942766 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:17.944768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:17.997804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.016992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 203/403] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:18.018057 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.039957 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.041661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204573-ea43-41f5-bd4f-06bd818b287a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.077104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.077734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.078012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.078281 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.079442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.080879 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.087432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 201/404] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:17 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:18.087432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.151759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108175e8-5359-49c4-b52a-2d8be93418ce None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.188659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.189639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.190118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.190603 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.191945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.194142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.200550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 204/405] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:18.201659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.207097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e6c1-e74e-44e2-b744-49a163232aa8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.249416 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.249992 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.250270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.250542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/groups/5418e8fe9f344091ae36cb24a376db36/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.254962 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a48b5026b743e7bfdce3491cbe8f30, group_id=5418e8fe9f344091ae36cb24a376db36, role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.256752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.260602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.275178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 202/406] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 03:16:18 2022] PUT /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30/groups/5418e8fe9f344091ae36cb24a376db36/roles/4363620f6a3647d0a382928ffef15ec7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:18.276534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.314702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.343388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf961-5f70-451f-a642-e362436123c6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.392979 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.394552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.395077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.395620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.397187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.399007 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.406481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 205/407] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:18.408096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.423125 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455f8c4-0545-4026-a2d7-79bae3a90113 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.466330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.467039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.467357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.467636 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.468965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.470549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.477039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 203/408] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:18.477039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.538762 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1913c-2cfc-4292-9449-685ed8fdb062 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.549280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.581185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.581818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.582097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.582374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.590742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98f907-6f2e-4186-85bd-85751e2ab5e7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.594936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:18.601286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:18.602184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:18.602975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:18.604006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 206/409] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:18.604370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.620071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.633481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.634507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.634507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.634673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/groups/5418e8fe9f344091ae36cb24a376db36/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.636862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, group_id=5418e8fe9f344091ae36cb24a376db36, role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.638441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323a25-6174-47c0-952f-2d2a1acc0dd4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.658967 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 204/410] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 03:16:18 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/groups/5418e8fe9f344091ae36cb24a376db36/roles/4363620f6a3647d0a382928ffef15ec7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:18.661243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fd88-a569-48d3-87d9-9a39b75c2bb5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.661718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fd88-a569-48d3-87d9-9a39b75c2bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.662220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fd88-a569-48d3-87d9-9a39b75c2bb5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.665058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 205/411] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:18.666122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7fd88-a569-48d3-87d9-9a39b75c2bb5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.727944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62204b7-15b6-4f56-8be3-dfb0f3f92f35 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.767270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.768260 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.768751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.769218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/groups/5418e8fe9f344091ae36cb24a376db36/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.771520 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad54fcbcadc44dd8177cacbea9c897d, group_id=5418e8fe9f344091ae36cb24a376db36, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.773257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6e854-802f-401b-8c14-ec122132ca20 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.789526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 207/412] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 03:16:18 2022] PUT /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d/groups/5418e8fe9f344091ae36cb24a376db36/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:18.791168 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.791277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7fd88-a569-48d3-87d9-9a39b75c2bb5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:18.791733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.792188 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.813259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:18.814799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:18.820054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.856950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:18.857832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:18.858349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:18.858722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:18.859703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:18.860107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:18.861900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:18.892092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 206/413] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:18.898341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:18.910760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:18.922061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 208/414] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:19.018412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906f5e-be67-4a2a-867b-2a92effd7d3d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.064702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.065533 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.065819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.066094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] PATH_INFO: `/v3/groups/e5645567b5624d7dbf905edbf58098bc` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:19.067462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] RBAC: Authorizing `identity:get_group(group_id=e5645567b5624d7dbf905edbf58098bc)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:19.069008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:19.070869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 207/415] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:18 2022] GET /identity/v3/groups/e5645567b5624d7dbf905edbf58098bc => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:19.076336 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:19.201400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0251b-f7be-4fcf-b290-0c9d1ef03d54 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.244867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.245542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.245828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.246106 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] PATH_INFO: `/v3/projects/0c792b1b39084f70bab95471acabbdd7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:19.247372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] RBAC: Authorizing `identity:get_project(project_id=0c792b1b39084f70bab95471acabbdd7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:19.248877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:19.250810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 209/416] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:19 2022] GET /identity/v3/projects/0c792b1b39084f70bab95471acabbdd7 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:19.257004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:19.262897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:19.330506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:19.387742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5e9df-4d32-4142-a17d-922c7277c384 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.390018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:19.436996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.437678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.437959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.438240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] PATH_INFO: `/v3/projects/0c792b1b39084f70bab95471acabbdd7/groups/e5645567b5624d7dbf905edbf58098bc/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:19.441552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c792b1b39084f70bab95471acabbdd7, group_id=e5645567b5624d7dbf905edbf58098bc, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:19.444452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:19.469591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 208/417] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 03:16:19 2022] PUT /identity/v3/projects/0c792b1b39084f70bab95471acabbdd7/groups/e5645567b5624d7dbf905edbf58098bc/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:19.476498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb41ea-c7d6-40b3-a13e-59433876c24a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.527202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.527992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.528279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.528552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:19.546642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:19.551956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:19.554078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:19.554935 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:19.559071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 210/418] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:19.561135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:19.723596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b981a-8b1f-494d-9a7c-d1ddfbe965e5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.741354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:19.772197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.772882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.773169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.773439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:19.774115 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:19.774429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:19.775833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:19.789223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 209/419] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:19.794879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:19.945529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e0945-897a-49ea-84ce-7056cc3392b7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:19.981464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:19.991569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:19.992517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:19.992996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:19.993462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.086539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:20.088454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:20.226562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 211/420] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:20.229838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:20.390238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9b620-4424-4775-906f-5a9998529be8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:20.440000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:20.440635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:20.440914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:20.441189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.447827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:20.449280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:20.451079 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 210/421] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:20 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:20.456504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:20.508777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:20.615695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeae45-4232-4a30-996d-92762211c231 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:20.661298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:20.661924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:20.662311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:20.662592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.663776 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:20.665221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d37f6f-4c84-486d-a573-b6f80e0c122c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:20.671535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 212/422] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:20 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:20.676219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:20.694742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903a87-9ce4-4987-96fa-237b4499e130 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:20.695026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903a87-9ce4-4987-96fa-237b4499e130 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:20.695327 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903a87-9ce4-4987-96fa-237b4499e130 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.696840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 213/423] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:16:20.702339 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:20.702840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:20.703375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.723095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:20.724711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:20.793972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:20.826152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 214/424] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:20.885406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b141bc-ddbf-4cd6-b183-288dd09ca526 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:20.941742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:20.942330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:20.942617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:20.942886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:20.944167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:20.944476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:20.945906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:20.955713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 211/425] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:20 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=800a18a269264faba7cbec11735ac79b => generated 611 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:21.184411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:21.219218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:21.314400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6dba4-3aba-4466-80ee-93c7c73e8aff None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:21.353307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:21.353976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.354258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.354531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.370097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-eef43d0c-f190-424d-9fb9-de341f797192 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:21.375189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:21.376093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:21.376875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 215/426] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:21.382452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:21.497058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28ec04-09f5-4db4-bb30-4c798b1a498c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.497360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28ec04-09f5-4db4-bb30-4c798b1a498c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.497645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28ec04-09f5-4db4-bb30-4c798b1a498c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.499183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 216/427] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:16:21.504262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d385-f342-4deb-8a51-1eaa9ea2938b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.504767 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d385-f342-4deb-8a51-1eaa9ea2938b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.505242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d385-f342-4deb-8a51-1eaa9ea2938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.512371 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7718c-dc4f-4bc6-8fd5-c083ea3f8d74 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:21.518856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:21.545076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f509d385-f342-4deb-8a51-1eaa9ea2938b None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:21.547653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f509d385-f342-4deb-8a51-1eaa9ea2938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:21.547779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:21.548300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.548579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.548854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.549524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:21.549831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:21.554214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:21.562398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 212/428] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:21.568748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:21.633648 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 217/429] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:21.714069 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c58f9-935d-43a3-9056-d93c4941e83d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:21.754830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:21.755398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.755682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.756494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.801037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:21.821209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e88a-32f6-4d99-b1c9-69b424a06ed3 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.821496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e88a-32f6-4d99-b1c9-69b424a06ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.821773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e88a-32f6-4d99-b1c9-69b424a06ed3 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.827274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 218/430] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:21.831169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:21.831169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:21.831169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:21.849223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:21.850658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:21.851284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:21.853868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:21.871700 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:21.955329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 219/431] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:21.963792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.006545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 213/432] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:22.012249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.084117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e786c3-8eba-4081-a638-ba1d93e8f56e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.088385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.119688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443adbc4-03de-489e-958a-f0a0504bcd43 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.119982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.120621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.120905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.121180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.136821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9decca26-7a49-4c8c-b5f7-bd874ac5f0b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:22.141675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:22.142811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:22.143654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 220/433] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:22.144326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c73b-1066-45c0-8998-239bb7b7b08e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.144326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c73b-1066-45c0-8998-239bb7b7b08e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.144472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c73b-1066-45c0-8998-239bb7b7b08e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.146186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 221/434] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:22.152259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.152539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.152967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.160212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.162628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.163334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.163620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.163894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] PATH_INFO: `/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.165039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a48b5026b743e7bfdce3491cbe8f30)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.166940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.168767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 214/435] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/projects/d4a48b5026b743e7bfdce3491cbe8f30 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:22.169363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.174232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:22.175640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:22.247314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 222/436] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:22.248224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.290827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2a8e-3696-47de-b809-ddd134859943 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.327153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.327789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.328070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.328346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.329013 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:22.329318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.330695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.336218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.344663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 215/437] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:22.345433 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.355383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712e6ed-b9b1-4bd3-a24d-085752faaa44 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.396628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.397259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.397702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.397978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.399085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.400561 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.402109 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 223/438] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:22.402859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.410230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.450994 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4eaded-62b7-4ecb-a750-71d914777aed None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.487460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.488299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.488581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.488857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.499190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:22.504506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:22.505391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:22.506753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 216/439] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:22.507428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.522466 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa099e-6e07-415b-b956-ed34a700d5b3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.555930 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.556530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.556804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.557072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.564996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.605097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c151-b834-41f7-ae1f-23db9925d6c2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.623127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.624611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.626406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.638280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.638878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.639179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.639456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.640176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:22.640486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.641900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.647524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 217/440] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=d4a48b5026b743e7bfdce3491cbe8f30&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:22.652120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.738177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 224/441] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:22.739049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.767655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754be8d7-78e2-44bd-bc21-cf0b5378d287 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.796603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.809840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.810438 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.810711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.810915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.811625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:22.811935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.813349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.823731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 218/442] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:22.824638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.857952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e04d3-87bd-4280-8e66-0bd768540ae9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.860203 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:22.906836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.907568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.907845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.908121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.924804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:22.928818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:22.929734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:22.930516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 225/443] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:22.931199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:22.937398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34831f-d161-4a7a-840d-84fc71e268f7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:22.976158 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:22.976792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:22.977070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:22.977340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:22.978542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:22.980027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:22.983222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 219/444] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:22.983222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.051453 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.057209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34ed8-e39a-47ab-bcc9-2ff6afaaf5e7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.091926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.092461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.092739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.093013 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.098044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4322-cdbc-4495-b38a-dee9795e9cde None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.121320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.131908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.132508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.132789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.133059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.133719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:23.134022 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.135428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.143184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 220/445] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.143969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.159832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.161457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.242319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ef9b-e655-4cf2-94e0-d292d7a1745c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.257472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 226/446] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:23.258303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.258803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.279529 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.280168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.280452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.280729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.281850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.283312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.284861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 221/447] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.285623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.319565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.358563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6d9e0-299c-4270-be0c-46c31c1a604e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.382029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be55-0133-44a0-9645-5dc99753c100 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.397215 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.397912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.398262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.398540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.415129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.415801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.416081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.416355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] PATH_INFO: `/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.417467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad54fcbcadc44dd8177cacbea9c897d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.418886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.420509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 222/448] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/projects/7ad54fcbcadc44dd8177cacbea9c897d => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.421262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.464186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.465673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.476775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.522472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5537cea-3fd6-4a92-87b7-684bddb90e64 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.523495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.556282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.556887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.557169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.557443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.558165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=800a18a269264faba7cbec11735ac79b, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:23.558472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.559953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.562072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 227/449] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:23.562887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.570135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 223/450] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=800a18a269264faba7cbec11735ac79b => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.571268 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.682672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4a60-cc96-4299-91a6-64d8e0d3969f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.684849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c9ce8-e220-4995-8ec9-add4c7e6e57f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.701849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.722764 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.722981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.723520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.723686 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.723859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.723976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.724121 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] PATH_INFO: `/v3/projects/0c792b1b39084f70bab95471acabbdd7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.724218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] PATH_INFO: `/v3/groups/5418e8fe9f344091ae36cb24a376db36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.725408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] RBAC: Authorizing `identity:get_group(group_id=5418e8fe9f344091ae36cb24a376db36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.725507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] RBAC: Authorizing `identity:get_project(project_id=0c792b1b39084f70bab95471acabbdd7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.726916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.727012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.728786 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 228/451] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/groups/5418e8fe9f344091ae36cb24a376db36 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.731466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 224/452] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/projects/0c792b1b39084f70bab95471acabbdd7 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:23.734116 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.734264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.769706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:23.848886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cc09-7608-4a78-b486-3bc1bf9bfae9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.865978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329468b-f9b8-4edd-8f69-b5b81fb74def None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:23.889688 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.890688 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.890971 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.891273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] PATH_INFO: `/v3/groups/e5645567b5624d7dbf905edbf58098bc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.892433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] RBAC: Authorizing `identity:get_group(group_id=e5645567b5624d7dbf905edbf58098bc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.893878 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.895599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 229/453] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/groups/e5645567b5624d7dbf905edbf58098bc => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:23.902039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:23.905489 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:23.906111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:23.906387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:23.906668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:23.907452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] RBAC: Adding query filter params (group.id=5418e8fe9f344091ae36cb24a376db36, role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:23.907768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:23.910180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe29698-f495-4048-a217-0cacbb913e32 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:23.916223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 225/454] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/role_assignments?group.id=5418e8fe9f344091ae36cb24a376db36&scope.project.id=7ad54fcbcadc44dd8177cacbea9c897d&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:24.012641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:24.016981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0873ea-b9cf-48ac-aa4f-dff4a7dbeb43 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:24.052140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:24.053591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:24.053870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:24.054143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:24.054880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] RBAC: Adding query filter params (group.id=e5645567b5624d7dbf905edbf58098bc, role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=0c792b1b39084f70bab95471acabbdd7) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:24.055217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:24.056721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e408d-0b96-4bec-8579-e776f489d0f6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:24.063827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 230/455] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 03:16:23 2022] GET /identity/v3/role_assignments?group.id=e5645567b5624d7dbf905edbf58098bc&scope.project.id=0c792b1b39084f70bab95471acabbdd7&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:26.581881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7432f-2c69-4c21-95a2-a7188542d674 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:26.582391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7432f-2c69-4c21-95a2-a7188542d674 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:26.582463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7432f-2c69-4c21-95a2-a7188542d674 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:26.584044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 226/456] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:26.600813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9936-d77a-4fda-adf2-624b7b64f3fc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:26.601672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9936-d77a-4fda-adf2-624b7b64f3fc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:26.602097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9936-d77a-4fda-adf2-624b7b64f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:26.628218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-32fc9936-d77a-4fda-adf2-624b7b64f3fc None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:26.629803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-32fc9936-d77a-4fda-adf2-624b7b64f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:26.722855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 231/457] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:26.761368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed660-f526-4c7b-b2a2-2f1b191f1567 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:26.761652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed660-f526-4c7b-b2a2-2f1b191f1567 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:26.761929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed660-f526-4c7b-b2a2-2f1b191f1567 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:26.763494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 227/458] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:26.768923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332aaf0-a035-4c5b-b894-a754c0f3d2e8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:26.769226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332aaf0-a035-4c5b-b894-a754c0f3d2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:26.769510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332aaf0-a035-4c5b-b894-a754c0f3d2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:26.796196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4332aaf0-a035-4c5b-b894-a754c0f3d2e8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:26.797405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4332aaf0-a035-4c5b-b894-a754c0f3d2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:26.892237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 232/459] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:27.074238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ed660-f526-4c7b-b2a2-2f1b191f1567 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.086088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f37f2-dd54-4e56-995a-8ed1d463c719 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.086672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f37f2-dd54-4e56-995a-8ed1d463c719 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.087385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f37f2-dd54-4e56-995a-8ed1d463c719 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.089540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 233/460] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:27.095981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.096780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.097262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.118293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:27.120093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:27.139837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.155039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.244812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 234/461] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:27.266873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ed660-f526-4c7b-b2a2-2f1b191f1567 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.305953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.320715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.321361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.321746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.322032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.335924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.337385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:27.352833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:27.353765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:27.354669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:27.354669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.354669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:27.354669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 228/462] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:27.364133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.482977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228280-13e5-4a66-9f53-fa31b1835c5e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.529681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.530329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.530613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.530891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.543985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.545457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.554753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.557936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0f58b-cb9d-4036-bfbb-7a47c2bcb010 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:27.565642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:27.566807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:27.567768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:27.567768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.567768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:27.569714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 235/463] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:27.570850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.575196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.605576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.606189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.606476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.606748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.607430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:27.607829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.609524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.633610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 229/464] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:27.634600 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.705300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b38a82-e8f8-41b2-a0be-4ba575d19156 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.740390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.740945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.741225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.741506 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.751916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.753349 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.755417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53f4a-71e2-4496-818d-dd4188076acf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:27.767864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:27.768758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:27.769599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:27.769599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:27.769599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:27.769599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 236/465] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:27.769818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.804191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.804805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.805149 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.805429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.806075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:27.806383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.807795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f520e-1edf-41be-a129-9c5ecd9cd309 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.821847 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 230/466] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:27.823444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e478-034b-4f38-ad37-aacf1b8dda27 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.823961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e478-034b-4f38-ad37-aacf1b8dda27 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.824835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e478-034b-4f38-ad37-aacf1b8dda27 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.827170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 231/467] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:27.828815 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111e478-034b-4f38-ad37-aacf1b8dda27 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.831763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.854174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:27.878445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7849b-a9d8-4b03-b491-b367e894103d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.913475 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.914493 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.914790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.915067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.915747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:27.917866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:27.940741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 237/468] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:27 2022] POST /identity/v3/users => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:27.941617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111e478-034b-4f38-ad37-aacf1b8dda27 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:27.944711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:27.995879 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:27.996517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:27.996805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:27.997085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:27.998238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:27.998562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:28.000376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af34226-a0b1-4a8d-8480-80d0a8e03097 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:28.022791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 232/469] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:28.022791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.022791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.022791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.044664 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:28.046573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:28.087810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2740803-917a-483c-8a61-5e2f867928c0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:28.103989 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:28.135198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:28.135198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.135489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.135760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.136295 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:28.138236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f24d26-b9ca-4e00-848d-fe171b9c647a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:28.142421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 233/470] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:28.143050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:28.159258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:28.173694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 238/471] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:27 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:28.277911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c339d-605d-4db6-b61e-dc292beae083 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:28.325113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:28.325805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.326087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.326358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.327034 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:28.328607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959fe74-9edd-4f80-99fa-a3266d9b898f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:28.356829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 234/472] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:28 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:28.433187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f755348-8ffc-44b1-8797-94ea49cb8a0d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.433471 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f755348-8ffc-44b1-8797-94ea49cb8a0d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.433755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f755348-8ffc-44b1-8797-94ea49cb8a0d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.435250 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 239/473] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:28.440996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.441360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.441645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.462168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:28.463516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:28.522442 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f755348-8ffc-44b1-8797-94ea49cb8a0d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:28.541210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:28.558118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 235/474] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:28.650782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f755348-8ffc-44b1-8797-94ea49cb8a0d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:28.688552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:28.689285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.689577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.689843 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.701733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:28.703205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:28.718048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:28.719389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:28.720304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:28.720304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:28.720304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:28.720304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 240/475] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:28.727374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:28.837407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3344680-e9b3-4084-aee8-e4cd8522a1e1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:28.873723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:28.890259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:28.890888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:28.891189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:28.891470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:28.892126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:28.892435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:28.893816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:28.899269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:28.919155 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 236/476] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:28.926595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:29.021679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:29.052828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7a1d7-a876-4332-890c-2fc1c1665dce None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:29.096735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1ca52-4903-4613-8106-0bb83c24b622 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:29.116716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:29.117335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:29.117614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:29.117891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:29.118576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:29.119974 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:29.134153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.sql.core [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 03:16:29.134153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 03:16:29.134153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [parameters: {'id': 'f15dc079be85457a9d5891914daa46ee', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 03:16:29.134153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81380) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result.read() Oct 04 03:16:29.157721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:16:29.158620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application yield resource Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise self.value Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application session.commit() Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 03:16:29.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.session.flush() Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application compat.raise_( Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 03:16:29.160436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:16:29.161277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result.read() Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:16:29.162597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [parameters: {'id': 'f15dc079be85457a9d5891914daa46ee', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:29.164752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 03:16:29.166651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.168590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 241/477] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 03:16:28 2022] POST /identity/v3/roles => generated 144 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:29.168590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:29.188024 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:29.188664 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:29.188946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:29.189223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:29.189980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:29.191562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:29.237438 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 237/478] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:28 2022] POST /identity/v3/users => generated 249 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:29.270820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:29.289062 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342a1fd-25d3-42f7-8723-66539db6cfea None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:29.327024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:29.328058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:29.328560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:29.329047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:29.340137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f0726565-ab8b-48ce-85d4-99ab95b050cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:29.345671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:29.346605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 03:16:29.347468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:29.350806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 242/479] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:16:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:29.356835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:29.509352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdcda8-4cc7-45c3-ac23-39270c24f712 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:29.524094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:29.547983 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:29.548612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:29.548893 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:29.549170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:29.549850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:29.550210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:29.551680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790ef6d-53e1-4c60-afdb-e878bd6386f5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:29.566413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 238/480] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:30.683278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76824449-afd9-491f-abb5-094af3d756aa None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:30.683759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76824449-afd9-491f-abb5-094af3d756aa None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:30.683759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76824449-afd9-491f-abb5-094af3d756aa None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:30.685475 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 243/481] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:30.690659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304b73-3848-4c8e-9f00-059967baa9d9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:30.691072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304b73-3848-4c8e-9f00-059967baa9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:30.691279 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304b73-3848-4c8e-9f00-059967baa9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:30.713442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ff304b73-3848-4c8e-9f00-059967baa9d9 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:30.714613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ff304b73-3848-4c8e-9f00-059967baa9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:30.796145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 239/482] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:30.957227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc7fa-9a1b-4870-8db4-3b079ec9f46e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:30.957517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc7fa-9a1b-4870-8db4-3b079ec9f46e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:30.957689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc7fa-9a1b-4870-8db4-3b079ec9f46e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:30.959312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 244/483] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:30.964450 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f738d-2705-4c7b-a4a8-6a2d72c09f2f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:30.964736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f738d-2705-4c7b-a4a8-6a2d72c09f2f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:30.965068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f738d-2705-4c7b-a4a8-6a2d72c09f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:30.992092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a84f738d-2705-4c7b-a4a8-6a2d72c09f2f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:30.993272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a84f738d-2705-4c7b-a4a8-6a2d72c09f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:31.076748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 240/484] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:31.113076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bc7fa-9a1b-4870-8db4-3b079ec9f46e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.251504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bc7fa-9a1b-4870-8db4-3b079ec9f46e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:31.276139 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e080fa-89b2-4fe7-b537-fb4d0d4af5e4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.276449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e080fa-89b2-4fe7-b537-fb4d0d4af5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.276703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e080fa-89b2-4fe7-b537-fb4d0d4af5e4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.278306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 241/485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:31.285093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.285093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.285093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.301418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:31.302918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.303665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.304135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.305204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:31.306678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:31.326128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:31.331573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:31.332415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:31.333176 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:31.340607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 245/486] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:31.344567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.357392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:31.362663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:31.439879 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 242/487] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:31.440768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.543735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565497b5-a36d-4309-ba78-63946a8c95c3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:31.607971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79305d-6943-431f-b5af-52cbdc53d50d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:31.610890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:31.611526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.611807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.612079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.612745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:31.613051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:31.614502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:31.632243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 246/488] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:31.638150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.679661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:31.680355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.680641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.680914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.704546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:31.711628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:31.712479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:31.713252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:31.717869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 243/489] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:31.718712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.744270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:31.770605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:31.812740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6d1fa-7a59-4b75-8447-8a96ade2b49f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:31.852030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:31.852668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.853340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.853616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.866281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:31.867783 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:31.874574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def48db-24c4-4546-b7db-30465e13dbf5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:31.890495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:31.892416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:31.893997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:31.893997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:31.893997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:31.893997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 247/490] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:31.894275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:31.913383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:31.914169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:31.914553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:31.914929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:31.915848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:31.916263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:31.917815 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:31.926246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 244/491] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:31.927792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.017355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.026435 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b96d0-ca42-40e5-80d6-72eed0acf066 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.033496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd6d1-f075-4f5c-9c4f-a07e67ffecde None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.049128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.077429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.078110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.078232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.078629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.079026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.079364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.079792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.080215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.081028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:32.081843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.084482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.091897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:32.097299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:32.098412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:32.099607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:32.102260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 245/493] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:32.103371 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.106058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 248/493] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:32.109230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.222066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfb6f-4d99-4cd8-bea0-09809b2efd25 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.227919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a6786-d222-441f-95d4-58ac03e87d6a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.258978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.259614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.259919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.260191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.260844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:32.261153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.262526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.266829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.267501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.267779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.268051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.274840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 249/494] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:32.274840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.283879 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.285341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.292763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b2c1efe2-33b3-4380-8437-d7db84c2a940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:32.302154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:32.303370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:32.304314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:32.304314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:32.304314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:32.304314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 246/495] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:32.305350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc263-ad20-4e35-936a-0f91e240c9d2 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.305711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc263-ad20-4e35-936a-0f91e240c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.306051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc263-ad20-4e35-936a-0f91e240c9d2 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.311177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 247/496] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:32.311177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc263-ad20-4e35-936a-0f91e240c9d2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.322441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.381996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d19f9-59ac-4495-8731-0267277975ba None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.414845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.415503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.415733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc263-ad20-4e35-936a-0f91e240c9d2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.415828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.416054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.423402 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.424836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:32.437842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:32.438910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:32.439855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 250/497] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:32.440408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.453321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.453947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.454226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.454494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.464730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.466437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9049a7c8-d541-40b8-9a85-31741af0d85c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:32.481338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:32.482243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:32.483066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:32.483066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:32.483066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:32.483066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 248/498] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:32.483619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af83-9796-45c6-b104-a1a86d00b92d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.483863 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af83-9796-45c6-b104-a1a86d00b92d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.484134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af83-9796-45c6-b104-a1a86d00b92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.502411 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cfa5af83-9796-45c6-b104-a1a86d00b92d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:32.503654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cfa5af83-9796-45c6-b104-a1a86d00b92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:32.541224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.544494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b46870-ffab-44cb-b9e0-c604514d2921 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.549923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.570699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 249/499] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:32.572050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38bf66-948c-4029-a3bd-139f00a38e05 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.572350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38bf66-948c-4029-a3bd-139f00a38e05 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.572626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38bf66-948c-4029-a3bd-139f00a38e05 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.573868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 250/500] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:32.574566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38bf66-948c-4029-a3bd-139f00a38e05 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.587288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.587791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.588080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.588358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.589009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:32.589318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.590706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.607836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 251/501] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:32.607836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.684560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38bf66-948c-4029-a3bd-139f00a38e05 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.715412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b164f5f-bb46-4da9-bc0d-fb2233bb2c9f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.718806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.719444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.719724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.719997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.720731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:32.721068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.722515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9a051-cbb6-4cb8-8708-08b09a146295 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.753355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.754474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.754894 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.755368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.756688 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:32.757603 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.759398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.760393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 251/502] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:32.761693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.761985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.762260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.775792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 252/503] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:32.776785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.785941 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:32.787303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:32.800481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.819442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:32.858574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 252/504] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:32.859590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.900724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215a4a0-ac7c-4d7a-ab7b-465e426f4c5e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:32.937447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:32.938008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:32.938295 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:32.938595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:32.946512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:32.947973 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:32.960131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:32.961211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:32.962306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:32.963287 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 253/505] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:32.964167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:32.969081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e764d4-acb8-4eed-bd33-0d7850e7e74c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.004817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.006452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.006733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.007004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.044134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.048805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.071501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670146-5fba-4721-a19d-9e83e910b4e8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.080850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.082414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.106666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.107426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.107780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.108129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.115668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.117173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:33.128392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:33.129465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 254/506] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:33.131079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.186736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 253/507] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:33.187706 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.229539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241acb30-b022-4c8e-af3b-d8e20b3ef46f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.251860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.263552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e95eb-3624-4b69-8356-e395964842db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.264190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e95eb-3624-4b69-8356-e395964842db None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.264472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e95eb-3624-4b69-8356-e395964842db None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.264752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e95eb-3624-4b69-8356-e395964842db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.273079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.276285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e95eb-3624-4b69-8356-e395964842db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-116e95eb-3624-4b69-8356-e395964842db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:33.281984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:33.283053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:33.284549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:33.287325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 255/508] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:33.288006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcabf9-1e3f-432b-9eb4-aee00a0761a7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.288435 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e95eb-3624-4b69-8356-e395964842db None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.320763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.321361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.321645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.321912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.322623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:33.322926 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.324394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.341348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 254/509] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:33.341970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.389512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e95eb-3624-4b69-8356-e395964842db None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.422356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.422981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.423299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.423577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.435322 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.436785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.442668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9d81-97c3-4689-a06c-a9e72d5a1b97 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:33.451237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:33.452313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:33.453142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:33.453142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:33.453142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:33.453142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 256/510] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:33.453142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.469032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.478774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.479409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.479693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.479966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.480692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:33.480998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.482442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.500083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 255/511] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:33.500881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.507895 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.556326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8f62-83a9-4984-bd94-e4909bec2881 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.589621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.590247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.590525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.590799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.598454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1efeb-6d4c-4d51-be56-fc2cc4d4b914 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.601612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.603051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:33.617153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:33.618030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:33.618859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:33.618859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:33.618859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:33.618859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 257/512] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:33.619075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.634578 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.635208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.635488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.635757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.636412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:33.636711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.638068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.645774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 256/513] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:33.646895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.692299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.719871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380d72-9ba1-4ef9-b91b-a28d89192c4c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.722036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.744890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf230-3647-4fda-bafa-dd2cebe77e76 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.753071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.753685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.754209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.754209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.777391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.778197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.778479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.778751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.779431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:33.779738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.781113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.792867 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 257/514] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:33.793649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.823995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.825473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:33.890802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c261ac-f78c-4acb-ac5f-f255e6c23488 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:33.898413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.923826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 258/515] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:33.924784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:33.925840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:33.926451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:33.926732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:33.927007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:33.929638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:33.994404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:33.995803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.024022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed19aa-ec5a-4541-85f8-33205879c37e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.057759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.058690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.059061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.059628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.060393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:34.060830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.062175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.073299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 259/516] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:34.073871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.095357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 258/517] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:34.096165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.104558 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.136560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.175520 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ad39-4939-4766-b705-379d9e83dfe9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.194563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091819-4f7b-47a2-872c-0e4add172487 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.208341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.209505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.209788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.210064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.220603 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.222051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.230147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.230743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.231020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.231334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.231978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.233391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:34.236553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:34.237403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:34.238285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:34.238285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.238285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:34.238285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 260/518] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:34.238490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.253807 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 259/519] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:34 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:34.254732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.324259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.361106 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9abe-08db-40d9-babf-f64648a37afd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.371409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed862e2-d08a-4b3a-8907-420afa444e78 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.382430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.404524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.405186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.405469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.405748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.413571 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.414214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.414490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.414888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.419876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.421350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.425123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.426864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:34.437742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:34.438637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:34.439573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:34.439573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.439573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:34.439573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 261/520] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:34.440506 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:34.440700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:34.441498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:34.442501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 260/521] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:34.443107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.557389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78307ee6-321a-4b47-ac0d-2bffd78f5e07 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.557586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b53bf-4294-41c4-8c5b-577e74834245 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.580730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.594025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.594637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.594922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.595228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.595887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:34.596201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.596366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.596993 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.597252 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.597532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.597622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.608646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.609468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 261/522] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:34.610190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.614410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:34.625189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:34.626125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:34.626968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:34.626968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:34.626968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:34.626968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 262/523] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:34.627732 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.651188 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.718337 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093b49e-22fc-451b-9255-118a92190abd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.735247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d1c72-99e0-431f-8660-492a8621b66b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.752579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.753203 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.753488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.753768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.754449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:34.754760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.756205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.772128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 262/524] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:34.772128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.777688 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.778314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.778597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.778876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.779612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:34.779932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.781363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.805077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 263/525] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:34.805077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.818909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.878367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba272-ca2b-49a5-9717-9b9a702714b3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.890525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:34.912954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aeb14-2035-445e-a711-28062b75c394 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:34.914122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.914818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.915092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.915403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.923043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.924545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:34.938053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:34.938947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 263/526] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:34.940152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:34.954025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:34.954644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:34.954921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:34.955230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:34.955893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:34.956192 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:34.957564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:34.973861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 264/527] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:34.973861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.052794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f277-bc66-47e3-b2b1-bb69e2da4eef None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.054718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.081868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb632c9-1a81-422d-9ec7-d92cf5edf0b2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.089169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.090215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.090501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.090777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.098213 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.099813 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:35.111037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:35.111998 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:35.112864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 264/528] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:35.113320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.120863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.121476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.121763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.122043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.122774 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.123085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.124536 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.131251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 265/529] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:34 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.132335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.148892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.254311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266f21-9ae5-4baf-9c16-55d117304243 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.273836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d2ece-41af-4ba7-86b2-a85960efba35 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.297030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.298038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.298519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.299065 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.300207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.300722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.303332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.313369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.314102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.314391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.314723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.321886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.323416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.324772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 265/530] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.326225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:35.336883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:35.337919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:35.338934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 266/531] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:35.339514 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.354691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.431530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.438288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1e6b1-d07a-4536-bf08-2e50d4bd9122 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.475878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.476632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.477028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.477377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.478328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.478560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.480117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.481511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890ff92-3316-46e5-9d4d-b88810427568 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.503504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 266/532] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.504552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.520477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.521846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.522524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.522973 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.591971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.608447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.610332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.613761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621d61-3eaf-4c35-8ea0-88fabbe38980 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.650960 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.651736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.652110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.652479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.653305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.653660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.655479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.668854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 267/533] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.669732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.691814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.731898 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 267/534] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:35.733023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.782788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9295dd-5342-44bf-87c1-aefc66b2e058 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.821504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.821894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.822160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.822430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.823167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=7a7d32acd9af405c90b0c21fa8a20fb1) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.823482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.824924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.830740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 268/535] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/role_assignments?user.id=7a7d32acd9af405c90b0c21fa8a20fb1&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.831408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:35.834880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:35.862586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d5ad3-8858-4834-8bb5-8edf32ab9077 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:35.906088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:35.907020 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:35.907509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:35.907964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:35.908919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=447877c159084ece9e07080d0785e5f6) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:35.909423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:35.911105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:35.919901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 268/536] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/role_assignments?user.id=447877c159084ece9e07080d0785e5f6&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:35.970500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4fa52-2486-4bc1-8966-76399ee607dc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:36.021765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:36.022740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.023241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.023700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.037557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:36.039422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:36.058788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:36.060122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:36.061550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:36.061550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:36.061550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:36.063914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 269/537] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:36.072405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:36.113123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:36.197255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48e484-03a0-4e66-9d73-c902b38b65d7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:36.234045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:36.234897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.235397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.235820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.236665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:36.237118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:36.238685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:36.251969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 269/538] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:36.260412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:36.321362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:36.370619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c8756-d988-4da3-bdce-d92e777b1728 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:36.423917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:36.424882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.425316 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.425740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.433652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:36.435396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:36.454657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:36.455759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:36.456747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:36.458881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 270/539] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:36.464746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:36.585912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:36.604583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2843-e10c-48f4-975c-50e4acd31160 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:36.656460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:36.657003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.657286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.657566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.658270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:36.658578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:36.660031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d08d0-90b7-4ea3-bd2d-e2b89e402985 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:36.682450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 270/540] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:36.688361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:36.808552 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307ef4-0df8-4088-b441-4dc97cbc7a20 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.808831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307ef4-0df8-4088-b441-4dc97cbc7a20 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.809115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307ef4-0df8-4088-b441-4dc97cbc7a20 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.815246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 271/541] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:36.828808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.829098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.829385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.850412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8f499-648c-4d6b-abdc-8d4bb2778307 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:36.870711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:36.874065 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:36.888091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:36.923759 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:36.924616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:36.925048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:36.925474 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:36.926354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=dbf2caf4d02d44f19080cdc94f95ab4a) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:36.926868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:36.929575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:36.939307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 271/542] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:36 2022] GET /identity/v3/role_assignments?user.id=dbf2caf4d02d44f19080cdc94f95ab4a&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:36.981512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 272/543] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:37.260512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:37.365541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3331f-0831-4ecd-b669-934749a94cd2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:37.407638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:37.408274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:37.408556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:37.408835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:37.419332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:37.425552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:37.426405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:37.427249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 272/544] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:37.433091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:37.513704 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:37.538099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe4e41-288e-49c1-94b9-cb79a3e32ed3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:37.583922 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:37.584564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:37.584848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:37.585124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:37.585803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:37.586111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:37.587598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f31985-1ef8-4ae1-a278-845538ab85b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:37.598761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 273/545] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:37.604729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:37.695738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043086-9f3a-470d-93c8-afe6fb6faad0 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:37.695946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:37.696450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043086-9f3a-470d-93c8-afe6fb6faad0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:37.696802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043086-9f3a-470d-93c8-afe6fb6faad0 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:37.698569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 274/546] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:37.704106 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:37.704469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:37.704815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:37.735769 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:37.737148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:37.777739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341da3-5e7a-4f91-86e5-5e30e4491689 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:37.846682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:37.847788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:37.848275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:37.848929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:37.867608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 275/547] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:37.868979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:37.871016 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:37.886751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:37.887692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:37.888511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:37.888511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:37.888511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:37.890635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 273/548] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:37.900265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:37.996036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:38.031653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85b56-f265-4723-b879-bdecf023ae5c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.079200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.080286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.080770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.081300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.082164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:38.082670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:38.084845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:38.097453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 276/549] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:38.102661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.234595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:38.243659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ac8b-935b-4056-bc68-8ece1ad72c89 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.289006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.289564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.289844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.290122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.300208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:38.301675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:38.314657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:38.316067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:38.317110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 274/550] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:38.322101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.419490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.432218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f8c6-5ecb-4db7-8642-bae31cda1af7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.469663 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.470282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.470571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.470966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.471743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:38.472059 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:38.473502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:38.490785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:38.513742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 277/551] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:38.522600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.570373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca942d-c37e-4612-b5bd-064201bc66f0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.611764 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.613749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.614242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.615276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.629232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1e8bd42a-b2a1-4d6f-a541-4a2d9d49c12d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:38.636676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:38.637740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:38.638546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:38.640698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 275/552] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:38.643206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30897-479a-48ce-91d8-c9bd2b7cb08b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.643499 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30897-479a-48ce-91d8-c9bd2b7cb08b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.643778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30897-479a-48ce-91d8-c9bd2b7cb08b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.645366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 276/553] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:38.650939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c30897-479a-48ce-91d8-c9bd2b7cb08b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.656149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7a47d-cd46-4b9d-945e-5a94c13421ac None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.667540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:38.701269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.702264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.702698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.703228 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.760504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c30897-479a-48ce-91d8-c9bd2b7cb08b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:38.765228 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:38.779401 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:38.780850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:38.796898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:38.797500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.797791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.798046 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.798700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:38.799004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:38.800422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de1cb-5a05-4290-93b3-bac764d4c655 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:38.810553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 277/554] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:38.815184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0269c-07eb-431f-95bd-c68eee4e9223 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.815184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0269c-07eb-431f-95bd-c68eee4e9223 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.815184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0269c-07eb-431f-95bd-c68eee4e9223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.832006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1ef0269c-07eb-431f-95bd-c68eee4e9223 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:38.833133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1ef0269c-07eb-431f-95bd-c68eee4e9223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 278/555] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e9f0-c4d9-421e-9270-56cbe177378b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e9f0-c4d9-421e-9270-56cbe177378b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e9f0-c4d9-421e-9270-56cbe177378b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 279/556] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:38.904561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e9f0-c4d9-421e-9270-56cbe177378b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.905869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 278/557] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:38.907161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:38.925499 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.017675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.033272 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6086d-3987-43bd-88e6-5fc875a1fd6c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.041261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e9f0-c4d9-421e-9270-56cbe177378b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.080524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.081174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.081453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.081725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.084690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.085563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.086046 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.086659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.092734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.094683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.098098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.100323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f5f0389a-59a2-4234-a083-c238d2263cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:39.111373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.112354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:39.113278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:39.113278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:39.113278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:39.117155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 279/558] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.118399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.118898 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.119050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:39.121336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.122215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:39.123084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:39.123084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:39.123084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:39.129612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 280/559] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.131322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.138698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:39.139940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:39.196910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.230914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 280/560] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:39.231938 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.263703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929e67-60fd-4ff1-ad22-d68d9682921f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.303742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.304372 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.304648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.304918 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.305661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:39.305969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.307554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.321189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 281/561] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:39.321833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.323451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.366298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f7e39-753d-4050-8fbf-1c8f7446696c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.409765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.410675 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.411173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.411707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.412532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:39.413049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.414615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.427544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 281/562] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:39.429025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.454390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bcc7-69a6-4c05-b6f7-cd5ac8995922 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.476973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.505880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.506763 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.507070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.507762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.522894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:39.528797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.529685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:39.530492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:39.531778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 282/563] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.532940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.560403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a908eb3-e34c-4e97-bc75-845b00b76c14 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.594370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.601434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.602074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.602356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.602630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.609218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.610668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:39.621859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.622893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 282/564] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.623817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.636797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7b44c-8664-4c47-aeab-dcb0995468a5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.670064 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.670691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.670970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.671277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.678474 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.680004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:39.691709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.693317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 283/565] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.694399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.725088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e44f4-7cea-406c-8d74-884aa3e2f8fa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.731077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.758647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.759314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.759602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.760013 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.770059 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3f603652-df84-4702-9671-96828de6f9f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:39.775463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:39.776463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:39.777324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 283/566] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:39.778237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8da3-9f7b-4b26-ad2f-4952e4e55eca None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.778408 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8da3-9f7b-4b26-ad2f-4952e4e55eca None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.778682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8da3-9f7b-4b26-ad2f-4952e4e55eca None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.779884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 284/567] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:39.780581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d8da3-9f7b-4b26-ad2f-4952e4e55eca None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.797143 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f3f3-f6ca-4aba-9679-c3c04db0a4af None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.831475 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.832089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.832368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.832502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.832634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.833428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:39.833806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.835457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.843377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 284/568] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:39.844292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.885066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d8da3-9f7b-4b26-ad2f-4952e4e55eca None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.918655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.919359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.919651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.919929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.920620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:39.920931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.922385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:39.934411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 285/569] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:39.935393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:39.943520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:39.948098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c26eaf-c8e3-4adc-a6cd-76f8aea95a4d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:39.981343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:39.982040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:39.982387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:39.982729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:39.983559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:39.983868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:39.985280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c714-b74a-439d-85bf-0dd6cf097099 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.002265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 285/570] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.003283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.003570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.003841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.021607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:40.022793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:40.037360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7c5d-e23f-4f3b-b768-86ffee864ca6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.047655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.074263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.074864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.075158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.075440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.076091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:40.076396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.077746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.086148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 286/571] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.086915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.091924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 286/572] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:40.092985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.192000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0dc96-7e67-42b9-a72f-5226f4e9c7cd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.194547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.211349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cf0df-8b55-47e1-bc3e-3bcb635e0ff3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.228727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.229354 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.229644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.229916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.240791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.242224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.246519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.247415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.247900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.248384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.249233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=bb8d1e9116254e08849307aa3450df44) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:40.249675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.251235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:40.256161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:40.267416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:40.268582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:40.268582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.268582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:40.268582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 287/573] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:40.268870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 287/574] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/role_assignments?user.id=bb8d1e9116254e08849307aa3450df44&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.268959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.269040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.306628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.386779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b330-abbf-450b-b4f0-321a811c9d42 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.412734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc276d-dfc3-4792-8cfa-e53d9eeeb8c9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.427754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.429059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.429399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.429735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.443177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.444545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.450294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.450953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.451273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.451553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:40.460133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:40.461103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:40.461973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:40.461973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.461973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:40.461973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 288/575] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:40.462207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.465161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.467027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.468440 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:40.483724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:40.484884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:40.485841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:40.485841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:40.485841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:40.485841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 288/576] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:40.486071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.571293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.581846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845ecf-350e-4bbf-b3c0-a271834204a8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.595917 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947fd7c-7163-42ad-8f7c-621313d1abf2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.622971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.623857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.624139 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.624448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.625307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:40.625415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.626734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.633259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.633905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.634202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.634529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.639003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 289/577] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.639687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.649967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:40.655672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:40.656567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 289/578] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:40.657395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.711070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.751063 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485958-8379-4d4d-94b5-a4ad10bfcbcd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.764079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bbd8-c4b3-4933-b678-8d0e99b78617 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.785180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.785790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.786070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.786407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.787053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:40.787390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.788765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.799846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 290/579] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.800642 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.805857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.806486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.806772 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.807048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.807741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:40.808052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.809459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:40.812190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.823191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 290/580] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:40.824389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:40.926002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83ab39-cf0d-47a9-829f-59a2ad918bb8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.951848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d1526-8e45-4e07-8e8c-df9d5b317184 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:40.956690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:40.973509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:40.974444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:40.975104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:40.975548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:40.986354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:40.987955 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:41.001713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.002691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 291/581] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.004253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.008981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.009678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.010032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.010490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.011377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:41.011756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.013220 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.023174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 291/582] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:41.024387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.097795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.131000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b83912-470e-4486-b8ae-70197e16845b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.149217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c39b11-e3a8-44a6-9e9b-4891d4e3d1cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.178660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.179634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.180122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.180470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.187302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.188048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.188436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.188769 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.190593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.192137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.196453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.197864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:41.209495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.210359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:41.211641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:41.214683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.215596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:41.216527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:41.217060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 292/583] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.217060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.217182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 292/584] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.217772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.265000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.339667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745eac-33e4-4b46-8759-ece2698ee202 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.342726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fd564-113f-4450-a7d8-8b6ef338e3d1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.359645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.374355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.377648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.377648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.377648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.387251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.387865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.388145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.388425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.389155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:41.389460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.390498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.390878 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.392361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:41.407455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.408584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:41.409655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:41.409655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.409655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:41.410856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 293/585] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:41.411500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 293/586] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.411738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.412681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.518779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.523771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff251d-4135-4869-b01a-72e4451d4cc0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.526234 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94dad7-c3fd-4d05-9bc7-a64611998107 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.560232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.560560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.560853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.561137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.561233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.561412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.561506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.561731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.562101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:41.562410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.562513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:41.562735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.563890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.564181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.581265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 294/587] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:41.581265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.586093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 294/588] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:41.591183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.621803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.689511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d7294-94ab-411b-9e7b-876a9dce6c98 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.692302 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072e5e-dc43-4a89-92c5-4ef0c8c930b3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.725681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.726313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.726592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.726866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.727076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.727530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:41.727719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.727815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.727992 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.728263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.729169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.738285 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.740283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.745884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 295/589] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:41.745884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:41.760117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.761057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:41.761947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:41.761947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.761947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:41.761947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 295/590] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.761947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.766617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.848096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:41.855255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb0d6-5d2f-4b55-8f81-4ef1f8be202b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.879036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791329f1-733a-4ac7-8d17-2ec2a2cea3fa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:41.891551 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.892213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.892489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.892786 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.907988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=2c5eae01c70b440e95b480d51780dd3e, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.909555 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.914377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:41.915245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:41.915616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:41.915890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:41.923448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 296/591] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:41 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:41.924397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:41.927105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:41.928877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:41.947668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:41.948897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:41.949989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:41.949989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:41.949989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:41.953767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 296/592] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:41.954729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.038682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.067081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed55ed-d8bd-467e-8880-c3d6cc1bce6c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.108214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e970d4-0f84-4500-a721-7c7b3594e50b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.110556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.111742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.111742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.112687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.121465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.122923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:42.136816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:42.137997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:42.138931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 297/593] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:42.139913 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.149017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.161120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.161760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.162150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.162424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.163275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:42.163580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.165047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.180798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 297/594] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:42.181656 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.262224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678580f-b4ee-4a34-a562-dfc44236f994 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.294540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af0d-170d-4f10-ac4b-abb6dabb66d9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.301529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.303383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.303668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.303946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.304596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:42.305253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.306931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.323252 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 298/595] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:42.323834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.324883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.337042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.337731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.338014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.338293 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.339183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:42.339510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.341150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.362408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 298/596] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:42.363481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.411501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.452837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54899c-01c7-455e-b6f8-86eb7bff140a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.482629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ff22-e8c1-4910-aa88-5150e3811781 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.488446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.489080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.489386 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.489661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.496324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.497755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:42.509050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:42.509907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:42.510790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 299/597] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:42.511298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.530228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.531070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.531245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.531431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.539339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.540765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0689d2fb-9b0d-4051-8894-db10d7653a87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:42.552067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:42.552951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:42.554080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 299/598] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:42.554551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2755-1d96-41cb-a3d4-a51210715180 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.555238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2755-1d96-41cb-a3d4-a51210715180 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.555238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2755-1d96-41cb-a3d4-a51210715180 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.559257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 300/599] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:42.559257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2755-1d96-41cb-a3d4-a51210715180 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.595271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.629005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fcd7d-9f16-4a0a-b195-81c421fc6e53 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.657291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.668824 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.669463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.669750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.670026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.672075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2755-1d96-41cb-a3d4-a51210715180 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.681707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.683180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:42.699242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:42.700190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:42.701051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:42.701051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:42.701051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:42.702492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 300/600] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:42.703452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.712154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.712773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.713106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.713385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.714216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:42.714527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.716248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6599e0-bdb2-48ed-a996-d82baf122382 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.730263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 301/601] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:42.736456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.736746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.737020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.756101 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:42.757310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:42.804625 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380a21-a0b2-4d9f-b80b-c0f325587e4c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.831309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 302/602] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:42.831309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.851505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:42.852132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:42.852457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:42.852734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:42.853498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:42.853849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:42.855648 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:42.860773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.878734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 301/603] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:42.878734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:42.923208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:42.969880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208b10-4431-4f77-a877-8bb993a11f41 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:42.994672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da8c5-0729-4333-a4d3-fa4322afd3fe None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.014303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.015507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.015969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.016393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.017187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:43.017644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.020014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.035030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.035463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 303/604] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:43.036046 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.037042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.037401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.037535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/447877c159084ece9e07080d0785e5f6/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.053845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=447877c159084ece9e07080d0785e5f6, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.055538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.079178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 302/605] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:42 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/447877c159084ece9e07080d0785e5f6/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:43.079178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.152567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:43.178871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde3cd-59e1-4ac4-b12d-3a18dc3c71ae None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.187631 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:43.209066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57431ba-b863-4317-b35c-a68b274b566b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.230945 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.232235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.232716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.233183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/7a7d32acd9af405c90b0c21fa8a20fb1/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.251307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=7a7d32acd9af405c90b0c21fa8a20fb1, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.253147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.254694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.255462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.255745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.256017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.265095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.267406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.283532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 304/606] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:43 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/7a7d32acd9af405c90b0c21fa8a20fb1/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:43.284974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:43.291647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:43.292617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:43.293980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 303/607] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:43.299651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.467690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e35b78-f449-4e48-9e16-f02e470521f6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.505987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2d00-f08d-4aa4-a6d4-038c3312a544 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.515543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.516425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.517348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.517789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.520643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:43.539446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:43.544358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:43.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:43.546530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 305/608] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:43.551014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.562234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.562850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.563270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.563551 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.564242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:43.564548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.565959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.580932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 304/609] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:43.591166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.602436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:43.721665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ba15-c526-4bde-9500-1502fb2eee30 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.726922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe810ef-3447-44ad-a618-8b14acfb474f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:43.771633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.772258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.772541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.772817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.773492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:43.773800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.775213 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.791314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 306/610] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:43.797342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:43.811759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:43.812446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:43.812732 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:43.813014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/dbf2caf4d02d44f19080cdc94f95ab4a/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:43.836022 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=dbf2caf4d02d44f19080cdc94f95ab4a, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:43.837468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:43.863883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 305/611] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:43 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/dbf2caf4d02d44f19080cdc94f95ab4a/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:43.923241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:43.971343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e6313-0084-402b-98ff-021f6bac3071 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:44.015251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:44.016257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.016757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.017265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.034825 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:44.036743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:44.057027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:44.058055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:44.058928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:44.058928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:44.058928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:44.061141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 307/612] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:44.067320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:44.133803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:44.179693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de51d-4413-4fe9-8ecc-ecb9c9a947e1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:44.217166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:44.218158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.218660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.219172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.220037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:44.220558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:44.222144 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b458e-b074-489c-9df1-6e8116d8c4aa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:44.235037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 306/613] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:44.240919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:44.351894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ca86-4c5c-431b-bce8-6ef4bc2b805f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:44.361360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:44.404666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:44.405309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.405596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.405873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.420620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:44.422140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:44.426337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e529-c66b-4abf-9da9-b4b520fd8596 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.426641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e529-c66b-4abf-9da9-b4b520fd8596 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.426944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e529-c66b-4abf-9da9-b4b520fd8596 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.428612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 307/614] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:16:44.435868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.436156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.436417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:44.439830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:44.440947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:44.442094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:44.445262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 308/615] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:44.450814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:44.480163 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:44.482180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:44.560371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 308/616] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:44.603472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c1307-d569-4296-8b40-105947ae15a8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:44.638616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:44.639253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.639547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.639830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.640560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:44.640872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:44.642375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:44.661723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 309/617] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:44.667472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:44.723580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:44.771000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecf175-fdda-461e-8580-e05c65c64c16 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:44.806313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:44.806966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:44.807316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:44.807610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:44.818457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:44.819940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:44.834781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:44.835777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:44.836656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:44.836656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:44.836656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:44.836656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 309/618] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:44.841938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:44.882996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:44.917470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.014622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7138-1f8f-4348-b995-ee0bf5af570f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.045926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af417-80b6-4e9f-818c-c145816b0d4b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.051419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.052092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.052378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.052650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.053561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:45.053866 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.054247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:45.055270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.068715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 310/619] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:45.074884 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.101271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.101959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.102247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.102516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.132987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:45.138503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:45.139879 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:45.140703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:45.148012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 310/620] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:45.153705 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.169192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:45.189668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bad3-2d24-4fc7-8770-9484c4b8f5da None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.229667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.230305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.230585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.230862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.249239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.250714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:45.278021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:45.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:45.282254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 311/621] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:45.286368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.383006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368806-6d36-406c-9361-65582e3cc17a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.456202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.458526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.458814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.459093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.459797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:45.460104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.461484 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.474576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 311/622] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:45.479740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.486936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e9988-534f-4e82-8c8f-675ed14caa5f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.528509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.529276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.529514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.529716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.530466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:45.530776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.532430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.534095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:45.549250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 312/623] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:45.556364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.575954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:45.652239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4a30e-5338-493e-948d-e80d95a8ffbe None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.698693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.699456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.699736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.700058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.725080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.726477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:45.758981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:45.761183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:45.763306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:45.763306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:45.763306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:45.763680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 312/624] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:45.767903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:45.803884 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a307-a01f-4ae8-b2db-dccabc6445ce None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.845507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.846169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.846452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.846732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/bb8d1e9116254e08849307aa3450df44/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.863574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=bb8d1e9116254e08849307aa3450df44, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.865108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef043269-187d-4b10-89ed-f2a7b0c8a6d1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.871679 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:45.880954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 313/625] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:16:45 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/bb8d1e9116254e08849307aa3450df44/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:16:45.882733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b9b47-69ec-419e-a559-33a6062e238e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.883301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b9b47-69ec-419e-a559-33a6062e238e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.883824 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b9b47-69ec-419e-a559-33a6062e238e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.889990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 314/626] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:45.892359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449931-b5b5-447e-ad1e-8faf3c53b734 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.892891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449931-b5b5-447e-ad1e-8faf3c53b734 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.893394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449931-b5b5-447e-ad1e-8faf3c53b734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.902011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0652a18-a760-4bb3-a74b-020213ba5ab8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:45.913720 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-17449931-b5b5-447e-ad1e-8faf3c53b734 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:45.915391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-17449931-b5b5-447e-ad1e-8faf3c53b734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:45.946325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:45.947359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.948796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.949473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.952061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:45.952389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:45.953941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f1dc8-8ddd-491d-9dde-2d8b8837dbfd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:45.971615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 313/627] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:45.973362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983c088-8c3f-424c-a798-409056e7bea9 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:45.974429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983c088-8c3f-424c-a798-409056e7bea9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:45.974914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983c088-8c3f-424c-a798-409056e7bea9 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:45.976919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 314/628] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:45.979202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983c088-8c3f-424c-a798-409056e7bea9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.008301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.033032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 315/629] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:46.033032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.033032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.033032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.065174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:46.067012 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:46.131024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983c088-8c3f-424c-a798-409056e7bea9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.164402 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 316/630] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:46.177332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.178886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.179360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.179799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.182002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.189347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:46.191288 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:46.204337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:46.205525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:46.206730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:46.209491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 315/631] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:16:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:46.215618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.327589 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b047-3941-4f28-be6a-5a245063dffe None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.359084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.364331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.365072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.365561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.366453 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.367521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:46.368160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:46.370068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:46.389051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 317/632] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:46.397579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.420690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.569505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7147b-86ab-497b-b862-00d6be70b8ec None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.601049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ada0b-82ba-4f22-a021-be11adcef65f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.612508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.613144 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.613425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.613708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.624524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:46.629334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:46.630282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:46.631168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 318/633] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:46.631857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.640923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.641542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.641825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.642323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.668249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.716060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.724416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:46.725791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:46.757091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19581a02-7a3f-45ce-aaed-734106aeee03 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.794685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.795342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.795625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.796049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.809235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-439702a0-e9eb-4be6-b307-dccf6332c234 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:46.815830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:46.816740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 319/634] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:46.822997 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be858-a875-4f0e-b015-e9477dff3ee4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.823802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be858-a875-4f0e-b015-e9477dff3ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.823802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be858-a875-4f0e-b015-e9477dff3ee4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.823802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 320/635] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:46.823802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4be858-a875-4f0e-b015-e9477dff3ee4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.840391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 316/636] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:46.840391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:46.930454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.940119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:46.940762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4be858-a875-4f0e-b015-e9477dff3ee4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.947435 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baec18-bfb8-4f21-b166-66e556b6f793 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:46.979905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.980509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.980788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.981061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.981769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:46.982079 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:46.982190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:46.982727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.983011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.983338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.983493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c5d1d-2305-4a84-8cae-252b9149da73 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:46.984000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:46.984309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:46.985669 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:46.992465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 321/637] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:46.993890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:46.994189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 317/638] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:46.994526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:46.994928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:46.995036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.017052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:47.018476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:47.088645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 322/639] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:47.089875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.112424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c6a6-1d43-4fc9-9dbd-28eebc9a2e8f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.153389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.154028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.154315 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.154862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.169731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.171544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.188508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:47.191447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.193727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:47.194797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:47.194797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.194797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:47.194797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 318/640] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.194797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.209882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8b2b-f50f-4a82-bdb0-41f9c6286f28 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.210430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.248851 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.249923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.250437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.250958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.263704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.265479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:47.290857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.291870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:47.292738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:47.292738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.292738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:47.292738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 323/641] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.292738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.331731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab4b55-c23e-42a1-8099-6ce7f18367ff None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.383427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.384282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.384716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.385234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.396645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.398263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.407622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c27d4-8f84-4dbf-8bdb-e21735214f78 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:47.413553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.414515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:47.415428 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:47.415428 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:47.415428 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:47.416167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 319/642] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.417191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.444549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.445094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.445373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.445652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.446303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:47.446614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.448014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.454871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.460647 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 324/643] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:47.461726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.484840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.530122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2442c-cc9c-4f57-ade7-e8d0d988f165 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.567065 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9aba92-27e9-41c2-b2af-56679c925ab7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.567271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.567775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.568060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.568335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.569147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:47.570203 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.571609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.584045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 320/644] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:47.584904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.607165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.607797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.608083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.608358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.618917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:47.623793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.624687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 325/645] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.625645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.686109 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.697845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe7531-6697-439d-a4ee-8cc43a8c93ad None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.720685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.731095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a48c3-e3de-467f-8ee0-1bcb97423050 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.737063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.737683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.738139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.738849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.740063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:47.740377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.742125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.753883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 321/646] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:47.756735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.767162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.767791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.768067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.768331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.777750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.779235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:47.791044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.792080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 326/647] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.792965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.867605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27d14-c03d-4c19-8d04-94b7fa2323f4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.902759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.903433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.903720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.903998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.906785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4235690-7705-43e4-9659-34c2c995940b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:47.912334 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.913775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:47.925165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:47.926301 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:16:47.927629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:47.929413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:47.931293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 322/648] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:47.932181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.948432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:47.948980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:47.949267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:47.949813 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:47.950238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:47.950547 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:47.951940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:47.960715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 327/649] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:47.961514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:47.981132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.050122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f72c96-b9c9-46b9-9cd3-233b17af8f65 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.063812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4108-b854-4506-b02e-cb28ecc1be55 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.094476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.095348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.095740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.096026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] PATH_INFO: `/v3/users/glance` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.101290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.101924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.102206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.102483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.103221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:48.103539 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.105019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.105335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.107111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.123072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 328/650] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:48.123072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:48.123372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:48.124268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 323/651] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:48.125236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.168987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.231987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40c60-f9ed-4e9a-b5c5-6855cc2e890a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.236441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.241729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf1b6a-8326-4847-bd7c-b676bcd342db None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.265002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.265606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.265887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.266164 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.266888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:48.267228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.268663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.274691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.275360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.275650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.275927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.285870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 329/652] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:48.287048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.287255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.288537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:48.304197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:48.305356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:48.306270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:48.306270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:48.306270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:48.306270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 324/653] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:48.306270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.385542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.389712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7b39-d01b-4ed3-916a-d2974292f711 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.410150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481ec6e-359e-416b-af61-368dde593c8e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.424244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.424852 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.425133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.425405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.426169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:48.426501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.428749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0b7ff-f669-4311-a25c-2a1d3548cc9d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.434308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 330/654] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:48.435458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2bd4-4d8d-4819-a8ca-5434e3ae978c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.435736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2bd4-4d8d-4819-a8ca-5434e3ae978c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.436015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2bd4-4d8d-4819-a8ca-5434e3ae978c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.437200 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 331/655] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:48.437911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d2bd4-4d8d-4819-a8ca-5434e3ae978c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.450934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.451623 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.451908 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.452187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.453491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:48.453810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.455715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.480821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 325/656] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:48.480821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.493148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.558542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d2bd4-4d8d-4819-a8ca-5434e3ae978c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.596105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e53bb-6050-4a1f-ae9f-04c0e124c443 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.596984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.597771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.598219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.598806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.631886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.641738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.642359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.642642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.642917 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.643812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:48.643928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.645262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3acf1-cdf6-4492-8b8b-0e9eb29ce203 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.660205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 326/657] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:48.662660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.662958 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.663272 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.680228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.682038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:48.688585 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:48.689972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:48.755606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.761360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 327/658] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:48.762433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.808729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 332/659] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:48.809732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:48.882110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:48.892048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d23a-4c5b-434b-86da-4de29fee0cfa None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.928175 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5adb-ef16-4b1c-8c63-b1938c44d560 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:48.935815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.936522 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.936871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.937216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.971971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:48.973074 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:48.974822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:48.975304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:48.987037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:48.988932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.009578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:49.011360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:49.012267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 03:16:49.013204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:49.013736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.013813 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 333/660] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:49.014686 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.014783 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.126147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 328/661] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:49.127121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.136481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877043a-9faa-4cde-9af8-63d0c806c562 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.170948 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.184996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.185894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.186361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.186891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.198987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.200829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:49.216087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:49.217257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:49.218343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:49.218343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:49.218343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:49.223626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 334/662] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:49.224501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.233915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.238489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82d59-8eeb-47f5-bbed-ae9ea33824cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.278100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.278646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.278926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.279588 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.279958 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:49.280267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.281693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.304539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 329/663] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:49.304539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.330844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75412204-44c7-47c8-8fb0-1f353b37b8b0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.366027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.366660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.366940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.367237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.377563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:49.383730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:49.384691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:49.385552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 335/664] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:49.386238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.424929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adc369-bf1f-4fee-ba62-7db96550f5ef None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.425292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.462882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.463545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.463820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.464242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.477444 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.478663 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:49.494130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:49.495048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:49.495986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:49.495986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:49.495986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:49.495986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 330/665] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:49.495986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.501824 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e9a3-50e0-4bf8-b2db-75ca7638e9d5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.511040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.538553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.539090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.539391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.539660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.540297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:49.540597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.542379 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.554671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 336/666] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:49.555500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.608310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf5b56-b013-4a17-9e6c-fd93b9dc089d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.644498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.645509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.646016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.646495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.656082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.664861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3fc0-25bb-4e04-aec6-0030b9375a7b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.701528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.702133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.702407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.702676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.703361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:49.703665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.705029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.713491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 337/667] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:49.714345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.726233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.728025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.739214 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.822051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb311e4e-5cdb-47bb-af20-1c2976cc5599 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.840365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 331/668] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:49.840365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.858617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.859256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.859543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.859888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.860524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:49.860832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.862236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:49.874184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 338/669] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:49.875290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:49.894596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.950531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6fe98-11d8-4532-94c4-7e63ff28e524 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.962200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:49.979380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eab1-5f9a-4936-9863-b688d2ce31e1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:49.986671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:49.987440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:49.987785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:49.988065 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:49.995836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:49.997263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:50.009936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.010915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.011854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:50.015889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 332/670] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.016709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.021805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.022446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.022729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.023006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.033276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.035283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:50.050186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.051177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:50.052555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:50.052555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:50.052555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:50.052555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 339/671] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.052817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.124495 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.131952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e5a2c-9a11-4f56-9a84-744a2d3b301e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.162536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a6861-3884-4bb6-9a0d-5b5db86f44df None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.170274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.170987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.171712 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.172063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.183403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.185058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.197388 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.198019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.198297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.198567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:50.200577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.201658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:50.202755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:50.202755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:50.202755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:50.203502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 333/672] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.204347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.211878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.213342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:50.227583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.228473 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:50.229349 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 340/673] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.229836 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.233669 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.317817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee858d-a42f-4ab9-b021-acb8e781580e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.339840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb28afa-3243-480d-be41-be69284fdcc2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.355570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.356185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.356467 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.356742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.357430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.357736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.359168 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.367423 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.378048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 334/674] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.378685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.380986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.382055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.382055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.383135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.383135 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.383439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.384871 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.397114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 341/675] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.397837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.474456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.493823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bc17-bc78-415c-b559-03de8dfb7539 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.505739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ea58a-9c57-408d-89d8-b5cff559908f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.529943 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.530685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.531072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.531488 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.532246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.532691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.534179 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.542898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.543570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.543857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.544137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.544856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.545175 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.546713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.553652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 335/676] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.553652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.570244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 342/677] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.570244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.624031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.663309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2b6d9-e48e-4f06-8571-8d7f8a170df9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.677613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848921-0e65-4da3-b010-79754c601ec0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.696780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.697392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.697674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.697955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.698694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=447877c159084ece9e07080d0785e5f6) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.698998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.700526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.706323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 336/678] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/role_assignments?user.id=447877c159084ece9e07080d0785e5f6&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.706989 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.715235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.715890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.716168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.716439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.726029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.727058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.727573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:50.741040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.742260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:50.743235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 343/679] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.743854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.839924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5a8e6-07c3-476d-921c-4252914d38e6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.856124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224118b-8790-4a5e-9eeb-a9f54abb6c69 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:50.866118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:50.881346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.881989 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.882276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.882545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.889960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.891731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.899114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:50.899974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.900367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.900636 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.901265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=7a7d32acd9af405c90b0c21fa8a20fb1) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:50.901581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:50.903018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1e168fdc-3b40-4f0e-b6ff-044cfb87e9a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:50.905531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:50.906432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:50.907681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 337/680] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:50.908409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b6a3f-569c-4aef-adab-402bf349f28d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:50.908409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b6a3f-569c-4aef-adab-402bf349f28d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:50.908409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b6a3f-569c-4aef-adab-402bf349f28d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:50.909197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 344/681] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/role_assignments?user.id=7a7d32acd9af405c90b0c21fa8a20fb1&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:50.910525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 338/682] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:50.910641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:50.916031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b6a3f-569c-4aef-adab-402bf349f28d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.008396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:51.047215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5246-fd39-4d9a-a965-6d6fec685f4d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.054246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b6a3f-569c-4aef-adab-402bf349f28d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.099437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.099880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.100161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.100551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.101189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.101311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:51.101528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:51.101807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.102090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.102365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.103039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:51.103133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:51.103398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:51.105004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db065492-770c-462a-a1f2-5dab63965c57 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:51.119346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 339/683] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:51.120734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.120963 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.121238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.123656 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 345/684] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:16:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:51.124713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.142710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:51.143947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:51.168036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:51.233563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 340/685] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:51.239296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.258152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c5f1-c2c5-4283-97fc-3f3af68cb4a6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.311518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:51.314709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.315562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.315675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.315876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.316614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=dbf2caf4d02d44f19080cdc94f95ab4a) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:51.316930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:51.318428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:51.332385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 346/686] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/role_assignments?user.id=dbf2caf4d02d44f19080cdc94f95ab4a&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:51.363697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba00ca-d906-4d8e-b3f5-5bf4bb5cbd6a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.449568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.450117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.450403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.450680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.548010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:51.549452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:51.611722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:51.631732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.667197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 341/687] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:51.672619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.760524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:51.774328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9ae0-cd8d-4278-8837-adf75055df1d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.822923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.823980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.824496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.824970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.831716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7179d-6317-4fc2-8217-b64ea2013713 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:51.836284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:51.841776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:51.842685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:51.843553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:51.846172 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 347/688] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:51.861038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.886457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:51.888043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:51.888328 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:51.889411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:51.909221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:51.910696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:51.940886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:51.941859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:51.943753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:51.943753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:51.943753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:51.943753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 342/689] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:51.951792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:51.971477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.041466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a390a23-fbe0-417e-8363-aed940822f2d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.100574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.101200 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.101477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.101746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.103440 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:52.103752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.105129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.125948 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 348/690] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:52.132199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.137004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7a0bb-6afe-4dc7-9e33-6250bfb393d8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.156038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.218487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.220724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.221017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.222174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.222817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:52.223133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.224587 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.258303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 343/691] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:52.266485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.313274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c9a0-1a38-4135-af3e-35955491680b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.343112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.382339 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916b76-17fd-4d79-851d-2403e608ab1b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.396544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.397190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.397468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.397741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.420798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.421576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.421852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.422131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.424160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.425742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.429680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.431175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:52.444128 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:52.445028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:52.445915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:52.449999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 344/692] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:52.451198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:52.464410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:52.466552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:52.468625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:52.468625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:52.468625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:52.468625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 349/693] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:52.476097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.564992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d9d8d-5c72-46bc-a73e-d543e2fe4c47 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.615114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.630896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.631533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.631814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.632088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.632776 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:52.633081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.637165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.646233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.671047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 345/694] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:52.673611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.733275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77166bd-01a3-4de9-bc19-810bbcc21790 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.790305 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226600e-5f19-497d-b9de-94ee9fd2dbf8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:52.827319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.827853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.828134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.828410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.829242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=bb8d1e9116254e08849307aa3450df44) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:52.829563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.831041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.832443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:52.833778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:52.834269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:52.834715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:52.835572 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:52.836071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:52.837613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:52.843263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 346/695] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/role_assignments?user.id=bb8d1e9116254e08849307aa3450df44&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:52.867550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 350/696] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:52.872842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:52.944726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:52.998459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390d869-c927-48dc-b2b4-4fff44eabd94 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:53.036995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:53.037641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.037922 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.038198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.046181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:53.047674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5802220b-1936-4d2b-a696-e608df15b050 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:53.061426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:53.062409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:53.063372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 347/697] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:16:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:53.068462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:53.188252 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a2f5-b4e9-4292-9eed-3a97db52dfca None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.188440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a2f5-b4e9-4292-9eed-3a97db52dfca None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.188705 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a2f5-b4e9-4292-9eed-3a97db52dfca None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.190215 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 348/698] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:53.197539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.197826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.198100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.216842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:53.218029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:53.244510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd0da9-1da0-454f-933b-3f6986425b9e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:53.290219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:53.294365 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:53.295052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.296393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.296393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.296941 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:53.297326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:53.299315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:53.301843 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 349/699] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 03:16:53.319608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 351/700] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:53.324970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:53.396249 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:53.474042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c4bd-4a83-4aaf-8621-2c7c626e5ac6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:53.510693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:53.511342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.511626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.511901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.584375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:53.585903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848517b-7d9a-4df0-be00-250032caf632 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:53.609237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:53.637936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:53.701849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 350/701] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:53.703219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd901c-1f3e-4ae6-b616-806cceae949a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.703511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd901c-1f3e-4ae6-b616-806cceae949a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.703794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd901c-1f3e-4ae6-b616-806cceae949a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.705327 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 351/702] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:53.707448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd901c-1f3e-4ae6-b616-806cceae949a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:53.740714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f872-1786-4c4d-a3d5-c0e0e3fcdcf9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:53.778770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:53.779444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.779724 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.780000 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.780731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:53.782255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-32f56248-5b29-41c3-90c9-57d3822f40c3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:53.794134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:53.796522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:53.798648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 352/703] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:53.800600 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.800600 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.800600 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.819049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd901c-1f3e-4ae6-b616-806cceae949a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:53.819716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:53.821027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:53.854197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:53.854836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:53.855115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:53.855436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:53.866572 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:53.868066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:53.870680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:53.878562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:53.885874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:53.886896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:53.887837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:53.887837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:53.887837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:53.891307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 352/704] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:53.892325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:53.915192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 353/705] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:53.916535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.005054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae49e35-7ffc-456a-900d-58fe53886907 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.045308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.046100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.046520 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.046800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.047505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:54.047812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.049389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7527e-13fd-4342-b454-80eeb0c9bd4a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.059183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 353/706] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:54.061086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88b45d-5a06-4cb9-9af8-8ee2bf8d5256 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.061500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88b45d-5a06-4cb9-9af8-8ee2bf8d5256 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.061921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88b45d-5a06-4cb9-9af8-8ee2bf8d5256 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.067453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 354/707] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:54.067453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88b45d-5a06-4cb9-9af8-8ee2bf8d5256 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.077581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad89b-d574-4277-9e37-32edb6196cc4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.126867 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.128089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.128930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.129391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.130365 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:54.130887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.133312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cb96-d314-48ad-a462-c42c2a11e40c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.138659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.152076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 354/708] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:54.153380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.153673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.154010 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.172370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:54.173553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:54.197805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88b45d-5a06-4cb9-9af8-8ee2bf8d5256 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.209369 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.240237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.241232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.241989 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.242782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.243617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:54.244110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.245669 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.246510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 355/709] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:54.247487 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.254273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 355/710] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:54.255560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.379005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83485-96e8-488a-b729-ba702cf96a3f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.407469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.422964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.423664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.423946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.424534 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.440011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.441495 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:54.459635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:54.460851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:54.462147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:54.462147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:54.462147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:54.462147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 356/711] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:54.467819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.489597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440c398-6944-48fb-8463-3cd0e2def6e8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.531703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.532273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.532738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.533025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.540469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.541940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:54.554719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:54.555673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:54.556536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 356/712] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:54.557020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.591505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.625755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da42a3-251b-42bf-bae4-fa06fd402095 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.667011 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9fb7-983a-4926-8e91-e605f85f49fb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.671094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.671743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.672031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.672312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.672969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:54.673287 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.675178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.695237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 357/713] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:54.696135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.706835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.707508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.707856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.708137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] PATH_INFO: `/v3/services/network` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.709203 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.711320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:54.723108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:54.725922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:16:54.727500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:54.735168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 357/714] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:54.735168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.740875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.826263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60740e20-cbff-4f48-921c-cf5ba3912405 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.852528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:54.861292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36b65-601d-48e1-9ecc-e966fefe8965 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:54.868862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.869725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.870191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.870687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.871606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:54.872113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.873715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.886470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 358/715] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:54.887550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:54.911492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:54.912105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:54.912435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:54.912706 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:54.913460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:54.915628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:54.951355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 358/716] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:16:54 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:54.952671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:55.002113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:55.009988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158686a9-c1de-4555-8748-4ea8224c6d2a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:55.051901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:55.052529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:55.052811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:55.053341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:55.054108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] RBAC: Adding query filter params (name=network) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:55.054427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:55.057693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:55.061159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 359/717] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:55.070886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:55.156008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925907-fcea-4c19-8e7a-78331131b220 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:55.212482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:55.213120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:55.213401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:55.213678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:55.214413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:55.214751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:55.216526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934ade7-0fc1-4c0a-b6e7-12b30f2fd5cf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:55.227283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 359/718] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:54 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:55.256711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6e21-1718-4c1a-966f-4a1235465f63 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:55.296815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:55.297819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:55.298307 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:55.298802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:55.299730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] RBAC: Adding query filter params (type=network) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:55.300298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:55.302492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acefeb-01df-4eb1-b144-aafabd57d898 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:55.308936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 360/719] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:16:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:55.634085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aca3a-ff3b-4feb-a3b0-a28f19ff24f0 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:55.634410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aca3a-ff3b-4feb-a3b0-a28f19ff24f0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:55.634645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aca3a-ff3b-4feb-a3b0-a28f19ff24f0 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:55.636296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 360/720] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:55.641207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:55.642046 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:55.642533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:55.671496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:55.673041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:55.691387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:55.771484 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 361/721] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:56.169812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aca3a-ff3b-4feb-a3b0-a28f19ff24f0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:56.186760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:56.304661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aca3a-ff3b-4feb-a3b0-a28f19ff24f0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:56.346700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:56.347365 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:56.347646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:56.347920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:56.362961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:56.368381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:56.369276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:56.370103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:56.373963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 361/722] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:56.375361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:56.507568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20be92-96c4-4188-97c5-150b1b1166ca None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:56.560394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:56.561385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:56.561875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:56.562385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:56.563283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:56.563796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:56.565366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:56.576095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 362/723] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:56.581464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:56.706296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:56.745160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f33a3e-ca30-4511-8502-ec611e05951a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:56.780831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:56.781692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:56.782121 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:56.782567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:56.793360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:56.794972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1e9164ae-de92-48c4-a215-c617084829e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:56.810206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:56.811456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:56.812543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:56.812543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:56.812543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:56.814259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 362/724] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:56.820285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:56.841740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c804-447f-4279-95b4-c38b8d3c3aa5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:56.842029 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c804-447f-4279-95b4-c38b8d3c3aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:56.842300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c804-447f-4279-95b4-c38b8d3c3aa5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:56.843911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 363/725] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:56.848459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:56.849395 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:56.849512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:56.849773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:56.868080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:56.869337 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:56.950828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 364/726] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:57.006367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c881e8-9636-4a8c-a50e-70e3d7851162 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:57.059490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:57.060426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.060854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.061281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.062124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:57.062591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:57.064130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:57.081286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 363/727] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:57.086988 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:57.151497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:57.211366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e8e6a-f6af-41b1-bc0a-8455fe1f76db None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:57.258642 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:57.259309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.259597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.259870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.267320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:57.268785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:57.286989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:57.288036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:57.289595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 365/728] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:57.293814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:57.310405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:57.411312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:57.442657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cab7-4f15-435e-9fd3-1c6c1934282f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:57.497062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81eb91-275f-45f2-99aa-0fe47a30318e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:57.511077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:57.511727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.512010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.512285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.513074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:57.513393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:57.514811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:57.531492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:57.549706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 364/729] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:16:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:57.555977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:57.566128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:57.566684 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.567051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.567360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.568067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:57.569562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15c29-1f73-40a5-9440-ce6dabf61a12 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:57.596433 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 366/730] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:16:57 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:16:57.688546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bddda-9939-400b-99a5-72aba60aa26c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:57.697779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d4a16-35cc-4214-9460-dc2abe68f522 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.698104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d4a16-35cc-4214-9460-dc2abe68f522 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.698372 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d4a16-35cc-4214-9460-dc2abe68f522 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.700052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 367/731] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:16:57.705576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756e75-b16e-41ab-8ed7-fec016d23ea6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.705865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756e75-b16e-41ab-8ed7-fec016d23ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.706140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756e75-b16e-41ab-8ed7-fec016d23ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.730885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:57.745986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-35756e75-b16e-41ab-8ed7-fec016d23ea6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:57.749883 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-35756e75-b16e-41ab-8ed7-fec016d23ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:57.769453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:57.770444 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.771881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.772322 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.880642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:57.882219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:57.889057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 368/732] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:57.989705 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f319-a2cd-4a26-bcdc-b464cb4c6d9d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.990361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f319-a2cd-4a26-bcdc-b464cb4c6d9d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.990982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f319-a2cd-4a26-bcdc-b464cb4c6d9d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:57.992887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 369/733] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:57.997332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 365/734] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:16:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 03:16:57.999376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:57.999664 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:57.999942 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.002361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.020541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:58.021801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:58.033653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.118094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f50b-8ae1-47c2-9f30-c4ab3b6b515d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.145429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 370/735] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:58.155600 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.156245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.156531 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.156810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.168858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.170362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:58.188597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:58.189599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:58.191088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:58.191088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:58.191088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:58.191088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 366/736] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:58.195613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.224724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.258306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.314869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.341276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250875b2-51ac-4dd7-b9db-894e801f24e3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.364675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881f55a-1541-4f8c-a807-08a6779638d4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.384412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.385165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.385540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.386093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.405385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2456fe81-2b99-4b23-afd3-6b3632967e26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:58.409896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:58.411504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:58.412950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 367/737] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:58.413640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3999-a3a7-40aa-96cc-a41bba10f180 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.413640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3999-a3a7-40aa-96cc-a41bba10f180 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.413968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3999-a3a7-40aa-96cc-a41bba10f180 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.415303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 368/738] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:16:58.419954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba3999-a3a7-40aa-96cc-a41bba10f180 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.421081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.421927 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.422406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.422889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.423822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:58.424347 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.425973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a286a18-6f82-412e-aa88-fa8496e90f9f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.440043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 371/739] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:16:58.442271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.442771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.443297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.462634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:16:58.464309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:16:58.539761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 372/740] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:16:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 03:16:58.542982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba3999-a3a7-40aa-96cc-a41bba10f180 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.548041 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.578624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.579238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.579521 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.579792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.580449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:58.580750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.582102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.587307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.597319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 369/741] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:58.598924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.601201 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.689376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60db74b-4949-4859-b0b8-d13b7ca07bb6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.704964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc9d74-1cc9-4946-9b6d-682d44425076 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.740623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.741750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.741882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.742398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.742504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.742689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.742961 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.743377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.743649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.745170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.752008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.753697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:16:58.767212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:58.768141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:16:58.769023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:58.770980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 370/743] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:16:58 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:16:58.773719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 373/743] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:58.774946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.775101 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.875099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.883919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:58.900898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c46bd-9ab3-4bd0-8e5b-b570d044abf0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.948918 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bb656-8ed0-4acf-a539-839ffafcef01 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:58.954702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.955466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.955820 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.956171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:58.956937 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:58.959721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:58.960027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:58.979103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 374/744] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:58.979982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:58.996227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:58.996872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:58.997156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:58.997431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.011833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.013636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:59.031651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:59.032647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:59.033680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:59.033680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:59.033680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:59.033680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 371/745] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:59.033911 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.105869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc9fa-152b-4980-ae88-64461415dbfb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.141386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.142105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.142451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.142797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.153330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:16:59.158359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:59.159326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:59.161071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 375/746] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:16:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:59.162455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.171503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.189353 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76dad-f038-4848-a409-d7ab61506e53 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.202223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.235771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.236616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.236994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.237383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.238208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=bb8d1e9116254e08849307aa3450df44) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:59.238651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.240216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.247129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 372/747] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/role_assignments?user.id=bb8d1e9116254e08849307aa3450df44&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:59.248140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.279678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bd836-87fa-4927-b641-978967e8b607 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.318956 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.319873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.320310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.320736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.321545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:59.322037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.323661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.337306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 376/748] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:59.343004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.367441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346c643-7e02-4f57-944e-b91e8b62bfbf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.415126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.417676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.417676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.417676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.417676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:59.417676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.419253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.430095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 373/749] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:59.439318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.480519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e6cc7-de1b-43ac-954f-44afb3d3f52c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.516524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.536305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.536948 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.537228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.537501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.549122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.551207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:16:59.564799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:59.565756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:16:59.566672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:16:59.569925 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 377/750] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:16:59.575769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.647579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f1721-8407-4991-83f9-0c4da58cc5d1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.709225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.709878 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.710155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.710430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.729529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.731039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.749429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcdbc3-4f13-479e-9d2a-39269af4b2fd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:16:59.761021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:16:59.761977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:16:59.762885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:16:59.762885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:16:59.762885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:16:59.762885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 374/751] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:16:59.769390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.802220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.847868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.849295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.849576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.849845 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.850581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:59.850889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.852344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.872822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:16:59.887015 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01ab2-0e71-40c9-8fc0-4a282bc162ae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:16:59.895712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 378/752] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:59.902918 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:16:59.933018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:16:59.933640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:16:59.933928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:16:59.934207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:16:59.934853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:16:59.935182 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:16:59.936558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:16:59.950734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 375/753] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:16:59.956157 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.066117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ffdb0-bd90-4e43-8017-dbce6960e7d9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.109005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.109658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.109942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.110220 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.113808 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c5b9-c2ba-41de-b6c3-6c3042e59202 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.123123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.124619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:00.147611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:00.148611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:00.150156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 376/754] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:00.158213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.185704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.186367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.186649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.186922 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.187855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.206898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.209015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:00.226181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:00.227194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:00.228573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:00.228573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:00.228573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:00.228573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 379/755] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:00.233568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.270865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.350989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e2bbe-e4d1-4779-9a63-a781da340622 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.369270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976b35-4c19-491a-9043-cc58bce0ed93 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.391738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.391738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.391738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.391738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.392047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:00.392047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.392047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15538538-507e-465c-b9b7-8a6b65dee1fc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.402597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 380/756] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:00.408384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b11b-39d2-492f-9128-891c49598661 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.408669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b11b-39d2-492f-9128-891c49598661 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.408943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b11b-39d2-492f-9128-891c49598661 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.410307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 381/757] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:00.411794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2b11b-39d2-492f-9128-891c49598661 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.416469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.417084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.417363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.417636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.418365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:00.418669 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.420155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4076-a408-4504-abb3-aa4b8eb31d93 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.442277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 377/758] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:00.443584 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.443876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.444198 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.464064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:00.465592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:00.516161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.528374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2b11b-39d2-492f-9128-891c49598661 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.532258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.547169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 378/759] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:00.547169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.591370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.592353 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.592876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.593383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.607992 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.609719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:00.633276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:00.634199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:00.635062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:00.643599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 382/760] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:00.644670 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.649502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1f56-e8e3-4f3f-b9e8-7c433d259d14 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.690158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.690773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.692316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.692798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.780291 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3a621-096d-408a-acd7-337f46207bf2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:00.819995 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:00.820664 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:00.820946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:00.821274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:00.822014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:00.822322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.823833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.837931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 383/761] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:00.843485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:00.870796 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.878246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:00.887163 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:00.888670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:00.997311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 379/762] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:00.998172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.003224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297660bb-f1e5-417c-9011-7117328cd49b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.040432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.041462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.042017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.042497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.044797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=2c5eae01c70b440e95b480d51780dd3e, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.046569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.071465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 384/763] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:00 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:17:01.073210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.109640 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bc50f-9101-49d6-bcf2-f1cb69460929 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.111553 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.152889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.153591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.153870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.154190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.154866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.159472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:01.171185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:01.172239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:01.173056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 380/764] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 03:17:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:01.177299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.238246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.298409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fde20-be3d-4596-ab4d-5fd3daf76ee1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.339095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.339781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.340107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.340406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.353963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.356295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a61e0d19-5012-4851-a1bf-2eb219dbe7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:01.380632 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:01.381505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 385/765] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292f37-24d5-4b5e-8bd2-63ecec67beb3 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292f37-24d5-4b5e-8bd2-63ecec67beb3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.384518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292f37-24d5-4b5e-8bd2-63ecec67beb3 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.385278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 386/766] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:01.387665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e292f37-24d5-4b5e-8bd2-63ecec67beb3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.438889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce60c-4d36-46fa-a3fb-c1f949f00d29 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.475299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.475907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.476186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.476455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.477084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:01.477389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.478738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d1515-4923-4cf3-ada8-e8d40c338bde None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.483690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 381/767] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:01.484847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.485130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.485404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.501908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e292f37-24d5-4b5e-8bd2-63ecec67beb3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.508721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:01.509904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:01.527665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.528059 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.541549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.542389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.542820 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.543295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.544161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:01.544623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.546172 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.561307 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 387/768] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:01.562491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.585539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 382/769] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:01.586422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.673769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd9eda-5a1b-4f5f-a6e3-e37f4f364fb7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.710933 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.711805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.712263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.712685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.713461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:01.713914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.715461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5063a61-734a-475f-a184-83bd865f5ece None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.722186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 388/770] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:01.724114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a048f-eda6-4f1a-92f5-385773f2edeb None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.724552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a048f-eda6-4f1a-92f5-385773f2edeb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.724975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a048f-eda6-4f1a-92f5-385773f2edeb None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.726613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 389/771] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:01.730005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a048f-eda6-4f1a-92f5-385773f2edeb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.746276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95dbc6-33e7-4348-a114-45d3a2e201a0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.792359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.793469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.793754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.794032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.796743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.805370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.807601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1f32c897-ffde-4b04-831d-aa1bbfc2a879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:01.823644 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:01.824519 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:01.825383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 383/772] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:01.826327 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.826560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.826837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.851563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:01.853093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:01.861810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a048f-eda6-4f1a-92f5-385773f2edeb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:01.865200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:01.898101 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:01.898725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:01.899015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:01.899327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:01.900036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] RBAC: Adding query filter params (interface=public, service_id=c1897c4fa7c342598eaeb8786ba1f351, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:01.900356 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:01.901741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:01.914278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 390/773] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/endpoints?service_id=c1897c4fa7c342598eaeb8786ba1f351&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:01.914278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:01.936927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 384/774] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:01.944090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.022975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb7712-b406-4c49-9d16-54cd0af14de2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.057191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.057680 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.057960 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.058239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.058932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:02.059278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.060897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.063564 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.076488 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3531584-5bc4-4b83-814a-bf99581ca03f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.079099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 391/775] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.080376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.123543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.124940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.125425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.125892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] PATH_INFO: `/v3/services/network` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.127081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.128968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:02.141611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:02.142582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:02.143464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:02.146116 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 385/776] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:01 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:02.146901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.158527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.198314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6ff20-3991-4832-9dfb-4b8ee2720764 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.234717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.235789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.236168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.236533 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.237238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.238742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.244357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 392/777] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.245346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.275799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184cd01-f61a-4848-96c5-cc42532a477a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.285858 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.335387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.336315 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.336754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.337185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.338105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=36aa8a9229e84ff29445df661f752fcb) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:02.338591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.341202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.355418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 386/778] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/role_assignments?user.id=36aa8a9229e84ff29445df661f752fcb&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.357441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.402050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68edfc-7c1b-4759-a240-8a31637e0c80 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.476227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.477129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.477550 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.477982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.478765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] RBAC: Adding query filter params (name=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:02.479244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.481717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.482598 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.493774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 393/779] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.503106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.523388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972079d-2a3a-4fba-a74a-50b507b90b8c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.566998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.567676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.567955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.568229 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.592130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:02.598814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:02.599873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:02.600716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 387/780] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:02.608475 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.699613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.706375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4705ab-0e8a-42ef-b556-fc8556f4f772 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.753658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6c17-2758-4d4f-acf8-bb308737a4d7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:02.768977 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.769601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.770601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.770874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.771583 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] RBAC: Adding query filter params (type=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:02.771889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.773284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.786706 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 394/781] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.790152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.811748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:02.821300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:02.821853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:02.822137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:02.822415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:02.823097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:02.823432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:02.824798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:02.841366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 388/782] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:02.847702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:02.985946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2769f6-bc2b-4c88-8285-27d5a54c3540 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.023505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.024118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.024400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.024677 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.025378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] RBAC: Adding query filter params (interface=public, service_id=adf01b4a5e0646c69bceed5c845d40c6, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:03.025689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.027059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.029248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e19bc-e359-4f26-bcf6-ca747567ec1a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.033008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 395/783] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/endpoints?service_id=adf01b4a5e0646c69bceed5c845d40c6&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:03.043312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:03.073567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.074119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.074399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.074671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.097713 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.099724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.101426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:03.122827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:03.123845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:03.124723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:03.124723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:03.124723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:03.124723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 389/784] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:03.129961 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:03.193177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620e1f6-4880-4b3e-b77d-36520ee9af18 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.220424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:03.235450 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.236375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.236870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.237354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.238161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.239981 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.251645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 396/785] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:03 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:03.322345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02191db7-41fc-4351-a4a6-49ea53c02f06 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.388208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197555f7-6a48-4138-a226-0569f76874be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.388855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197555f7-6a48-4138-a226-0569f76874be None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.389133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197555f7-6a48-4138-a226-0569f76874be None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.389403 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197555f7-6a48-4138-a226-0569f76874be None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.390071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197555f7-6a48-4138-a226-0569f76874be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:03.390374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197555f7-6a48-4138-a226-0569f76874be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.391780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197555f7-6a48-4138-a226-0569f76874be None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.414469 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 390/786] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:03.420244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:03.559218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:03.568022 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4051cc5-5465-4014-a067-17cc546afc67 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.606634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0486-c77e-4ddf-a0f9-2dd560f43181 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.607271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0486-c77e-4ddf-a0f9-2dd560f43181 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.607749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0486-c77e-4ddf-a0f9-2dd560f43181 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.609187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.609815 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.610142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.610354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.610451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 391/787] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:03.617191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.617733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.618205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.618681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.620151 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:03.635288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:03.636961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:03.637906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 397/788] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:03.645638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:03.648204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:03.650047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:03.785117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd576f-8107-4ccb-bf7a-7a6e17f05c0f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:03.795617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 392/789] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:03.836711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:03.837320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:03.837597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:03.837865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:03.838593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:03.838898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:03.840356 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:03.868655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 398/790] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:17:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:03.874418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:03.937076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.029363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff67a-d31a-4814-8af8-14d7ee767172 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.073411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.074058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.074376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.074840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.089985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.091452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:04.109919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:04.110837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:04.111785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:04.111785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:04.111785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:04.111785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 393/791] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:04.118031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.138493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.172583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.312277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565144ba-4aa0-46cd-ba4c-b3c9fc5792e5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.339897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.368417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904644a-ce64-41f5-a1b7-943d9b9bfef7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.406446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.407616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.407909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.408169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.408819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:04.409124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.411080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.415172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.415806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.416084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.416355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.436766 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:04.443193 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 399/792] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:04.445055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:04.445975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:04.446799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:04.451615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.454594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 394/793] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:04.467450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.596827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.615635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f0a1-b683-49ed-a0b1-7c3ff59c8ae8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.632687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a248c4-2017-4397-a552-77519b48894d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.667414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.668050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.668331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.668606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.675532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.677206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:04.689166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:04.690036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:04.690915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:04.691367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 400/794] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:04.697419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.724042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.724665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.724948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.725226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.725899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:04.726210 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.727623 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.756779 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 395/795] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:04.759016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.774191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.880431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43179d55-f743-435b-8c1e-3a5a257cdce8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.918278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.919398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.919866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.920336 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.920923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae122d59-fa46-4e32-a7bb-332083ce9886 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:04.933056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.934742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9fe1d4d7-f29c-4b64-ae6b-6f717afcc86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:04.950718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:04.951722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:04.952606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:04.952606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:04.952606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:04.952606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 396/796] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:04.953302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f95f-4209-4bd9-8c36-28e113932486 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.953557 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f95f-4209-4bd9-8c36-28e113932486 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.953826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f95f-4209-4bd9-8c36-28e113932486 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.955345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 397/797] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:04.958102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0acbc4-5b6f-40ac-9df5-757f2969c7c8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.958381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0acbc4-5b6f-40ac-9df5-757f2969c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.958654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0acbc4-5b6f-40ac-9df5-757f2969c7c8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.960107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 398/798] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:04.960804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0acbc4-5b6f-40ac-9df5-757f2969c7c8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:04.962545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:04.970996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:04.971698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.971977 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.972251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:04.972982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:04.973288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:04.974703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aebb6-bfc4-44ac-b205-6ec60bdfb594 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:04.986797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 401/799] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:04.987978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab647a2-06f3-4761-9e27-bdddfb6a6852 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:04.988257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab647a2-06f3-4761-9e27-bdddfb6a6852 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:04.989053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab647a2-06f3-4761-9e27-bdddfb6a6852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.007051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cab647a2-06f3-4761-9e27-bdddfb6a6852 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:05.008278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cab647a2-06f3-4761-9e27-bdddfb6a6852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:05.035050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.083209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0acbc4-5b6f-40ac-9df5-757f2969c7c8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.083390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 402/800] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:05.083390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.083390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.083390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.100167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:05.101431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:05.128081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.129598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.130089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.130587 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.131489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:05.132155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.134506 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.152805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 399/801] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:05.152805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.178497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 403/802] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:05.179277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.230411 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.292747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778437d9-b126-4fef-93f4-0d5159e49edc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.306283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3bd73-785c-4126-9092-c6058c98df68 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.333858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.357692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.358361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.358647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.358925 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/bb8d1e9116254e08849307aa3450df44/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.361646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=bb8d1e9116254e08849307aa3450df44, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.363095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.369044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.369800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.370086 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.370361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.390954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.394058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.400040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 400/803] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:05 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/bb8d1e9116254e08849307aa3450df44/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:05.415066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:05.416119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:05.417044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:05.419709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 404/804] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:05.425242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.430479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.558384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc518f7-e236-418c-8d84-da9a86a8b29e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.571075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.610069 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.610728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.611080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.611381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.612866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044042-e097-4c7c-a29f-91cee99857b1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.627508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:05.634979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:05.635947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:05.636788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 405/805] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:05.642165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.656318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.656931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.657222 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.657482 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.658212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:05.658518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.659966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.684703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 401/806] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:05.685877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.742781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.767438 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddcfc9-d46e-41c0-90fd-c6cf6a4f7b9d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.805618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.806243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.806521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.806789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.807416 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.808838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.816809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddee9f4-8beb-4f80-b1fa-d16007a04d1a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:05.818523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:05.819553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:05.820368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 406/807] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:05.822476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.856128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:05.856800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.857673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.858200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.858492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.859187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:05.859493 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:05.860863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35684be-9694-4b95-a2c9-34fb03e681aa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:05.874314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 402/808] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:05.875684 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2261c-c908-44b7-8529-34f726dc3eb7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.875972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2261c-c908-44b7-8529-34f726dc3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.876247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2261c-c908-44b7-8529-34f726dc3eb7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.879549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 403/809] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:05.880810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2261c-c908-44b7-8529-34f726dc3eb7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:05.929522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc6e-16cf-4b46-93f4-48f48add884a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.964978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:05.965591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:05.966182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:05.966465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:05.993890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2261c-c908-44b7-8529-34f726dc3eb7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:05.995505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.032195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.032806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.033086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.033363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.033991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:06.034296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.036216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.041053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 404/810] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:06.041637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.041839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.043496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.154972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a29f-bffb-4f74-acf5-8248f58ece40 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.191232 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.192285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.192569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.192843 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.193436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.204683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.206131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-cba860a8-3d96-47a3-9bb6-95c1b0c44f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:06.223998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:06.224964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:06.225868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:06.225868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:06.225868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:06.227490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 405/811] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:06.229605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.230279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.230752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.254305 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:06.256041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:06.265330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 407/812] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:06.266345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.280865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.342780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 406/813] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:06.344050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.395726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e7f91-5f6c-44bb-8c80-c2d8ca2a55d0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.431598 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.432602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.433028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.433466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.434241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:06.434708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.436471 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.437970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.444106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 408/814] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:06.445165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.490462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00eb7b-9d77-418c-b33b-9382ca67f426 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.525972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.526842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.527382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.527939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.528789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:06.529321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.530876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.543465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 407/815] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:06.544642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.556856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.563518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c37a-c77a-4611-be48-0d7609b3c77b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.600126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.601222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.601716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.602295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.614110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.615893 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:06.634546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:06.635586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:06.636501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:06.636501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:06.636501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:06.636501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 409/816] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:06.637237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.668890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ede0-0dd1-4798-b66d-cf46a1b98f10 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.691315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.711136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.711715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.711991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.712376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.713041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.714392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.747173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 408/817] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:06.747173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.756534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728b96-37ec-4b87-b378-afab8484dee7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.804419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.805227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.805585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.806033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.815519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.817728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.832210 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:06.833615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:06.834615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:06.835907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:06.838422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 410/818] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:06.839526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.889718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c4e0-b151-4dd9-be0b-fbfa26612d06 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.928613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:06.929736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:06.930228 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:06.930739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:06.931628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:06.933218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:06.945807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:06.946804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 03:17:06.947714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:06.948788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 409/819] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:06.949594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:06.972385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7e525-03ec-4864-bb10-fbcaabfc445a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:06.982230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.009901 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.010510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.010838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.011114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.011779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.012086 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.013452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.029322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 411/820] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.030280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.079201 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31e3b-0c10-4ab4-87c9-1552ea974fd4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.110291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.120851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.122130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.123098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.123855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.124853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.125437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.127204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.146480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398216e-fe92-4280-b84d-14031fc59072 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.154520 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 410/821] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.155797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.185523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.186511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.186995 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.187514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.188372 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] RBAC: Adding query filter params (name=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.188882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.190456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.195810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 412/822] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.196876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.230092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.280120 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e80e9-1f60-44ac-98d8-a1c028c375b1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.324016 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28314237-20a7-4681-b3cc-392fbbd18b6d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.324744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.325785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.326269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.326822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.334758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.336618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:07.350238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:07.351577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:07.352504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 411/823] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:07.353266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.364270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.364898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.365175 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.365445 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.381408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.383312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:07.399906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:07.400915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:07.401787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:07.401787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:07.401787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:07.405854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 413/824] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:07.406671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.432697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.493978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.512341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2566d-0eb0-4b53-8fac-2691fc3cf983 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.513419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc378083-3766-4e20-895d-9af6dd19da95 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.553418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.554743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.555313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.555866 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.557054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.557691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.558258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.558967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.559369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.559888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.560298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.560687 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] RBAC: Adding query filter params (type=network) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.561065 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.562550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.573904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 412/825] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.575012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.578242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 414/826] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.580825 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.696929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec277a-c892-4d86-8087-996318b73131 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.702445 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e8c9-ecad-4b8b-86dd-03ad55d52d62 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.706939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.731293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.731908 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.732187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.732461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.733108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.733419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.734815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.741472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.743169 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.743677 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.744190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.745568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.747256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09da05-a83a-4e09-9794-915d334ed17a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.747841 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 413/827] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.748701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.766789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:07.779772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 415/828] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:07.781099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151973d-5ac6-4c44-bb8b-d792546743ef None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.781485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151973d-5ac6-4c44-bb8b-d792546743ef None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.781829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151973d-5ac6-4c44-bb8b-d792546743ef None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.795510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 416/829] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:07 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:07.796300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151973d-5ac6-4c44-bb8b-d792546743ef None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:07.883109 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72adbd-9c95-4ecb-8d37-6fe7e519cb45 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.940953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.941962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.942641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.943135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.944138 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151973d-5ac6-4c44-bb8b-d792546743ef None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:07.944420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:07.944899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:07.947795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c62a38-9095-4e5e-95aa-4906114a2f44 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:07.955798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 414/830] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:07.957838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.958264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.958674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.983377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:07.985477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:07.990887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:07.990887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:07.991310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:07.991401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:07.999552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:08.000995 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:08.013968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:08.015098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:08.016265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 417/831] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:08.021512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:08.023919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:08.083661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 415/832] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:08.099406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:08.147785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e021-1fcc-4c38-9efa-5fa13ba2fe80 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:08.186970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:08.187748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:08.188125 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:08.188491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:08.189299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:08.189702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:08.191250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:08.204850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 418/833] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:08.210351 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:08.314857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077981d-7f67-42ee-9ecc-01a7ec9ece82 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:08.349954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:08.350761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:08.351167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:08.351547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/36aa8a9229e84ff29445df661f752fcb/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:08.365956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=36aa8a9229e84ff29445df661f752fcb, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:08.367515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:08.375494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:08.393967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 416/834] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:08 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/36aa8a9229e84ff29445df661f752fcb/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 03:17:08.503484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325d273-2a5f-4f48-8513-d0c4321c9c8f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:08.524895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:08.545341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:08.546401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:08.546828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:08.547279 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:08.560733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:08.566015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:08.567162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:08.568252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:08.575311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 419/835] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:08.576848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:08.594847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:08.696308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3011-af6c-4ffd-823e-54ab3c56eebb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:08.733333 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:08.733946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:08.734227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:08.734500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:08.735185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:08.735497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:08.736846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:08.750233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 417/836] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:08.751652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:08.860488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54048e7-39da-4fa6-9d34-86d6a6418cf1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:08.905943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:08.906590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:08.906872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:08.907165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:08.918899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:08.920448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fd60c991-3c93-4158-a944-1637f5ab4d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:08.937579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:08.938559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:08.939492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:08.939492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:08.939492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:08.943246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 420/837] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:08.945293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.023576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.058012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a107-0f65-4984-b96d-933aa6dac14e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.094244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.094882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.095155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.095422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.096071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:09.096378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.098280 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.108170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338350b-f236-4d60-bb4b-f279ce650017 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.108465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338350b-f236-4d60-bb4b-f279ce650017 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.108868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338350b-f236-4d60-bb4b-f279ce650017 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.110808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 418/839] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:09.111258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 421/839] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:09.115979 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.120225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.120518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.120800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.139426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:09.140635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:09.166142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.219266 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 422/840] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:09.224390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238d78-b92a-4b60-bfab-793a8e394f23 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.255530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.261089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.261890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.262173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.262452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.270378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.271840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:09.283575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:09.284494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:09.285831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 419/841] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:09.292511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.417649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97c0be-3c86-4247-ac9e-0c2115bd6ae2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.453348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.453958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.454241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.454518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.455273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:09.455581 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.457542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.475481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 423/842] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:09.484244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.524768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.549591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.605485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79983ab1-8a09-4b1b-a9a0-0c4cceab09ef None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.645747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.646760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.647278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.647774 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.656613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e6e3-a62f-43df-b00a-a70f4295de3e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.708895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.710221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.710993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.711931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.712344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.714188 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.716630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.723184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.724631 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:09.730477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:09.731514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:09.732334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:09.735312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 424/843] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:09.741228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.830846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.832702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 420/844] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:09.839191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.855421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d084-1de6-49c6-9115-efdc95a5fdf6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.893552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.894300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.894749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.895036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:09.895819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:09.896129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:09.897765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:09.903015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 425/845] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:09.908057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:09.938535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:09.949789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227474ed-c2d1-413b-ac4e-7a4b063c8e99 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:09.989016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:09.989747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:09.990949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:09.991324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.003501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.005088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.023241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bd9eb-6a76-40a8-9288-bd6a5be29518 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:10.024273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:10.025173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:10.026067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:10.026067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:10.026067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:10.030150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 421/846] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:10.032326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:10.071331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.072322 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.072902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.073370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.074339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:10.074901 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.076746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.083071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 426/847] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:10.155116 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bffb-511e-4408-86dc-e1002ca9e30f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.194293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.194919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.195221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.195497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.196150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:10.196459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.197842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831d33a-f2bf-4294-bae1-0c5a804bc4b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.217009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 422/848] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:10.218507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:10.239781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:10.336039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c1147-ce0d-4811-944e-06721b973776 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.350454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833608f-a3d6-4928-9e7b-ac9c46258e34 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.352107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833608f-a3d6-4928-9e7b-ac9c46258e34 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.352107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833608f-a3d6-4928-9e7b-ac9c46258e34 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.352620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 423/849] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:10.357377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.357670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.357939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.378054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.378598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:10.379681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.380113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.380529 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.383654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:10.385371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:10.389161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.390773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:10.402609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:10.403725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:10.404810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:10.406870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 427/850] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:10.416830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:10.483187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 424/851] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:10.515285 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:10.548584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bc2dc-126e-46df-b09a-87f5d59f15b1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.583896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.584526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.584804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.585078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.585792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:10.586176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.587693 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ee160-4bd8-4766-b05d-fa4029bd7f31 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.603107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 428/852] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:10.609388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:10.708377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0143ad-ebd1-46d2-9b7f-d9b501655b1f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.708985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0143ad-ebd1-46d2-9b7f-d9b501655b1f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.709587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0143ad-ebd1-46d2-9b7f-d9b501655b1f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.711994 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 429/853] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:10.715383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa965-1f9f-4d0b-817d-c19c1de8c1a7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.717301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.717630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.718007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.736703 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:10.738026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:10.750508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.751338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.751756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.752194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.753061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=bb8d1e9116254e08849307aa3450df44) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:10.753527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.755122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.759440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:10.761850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 425/854] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/role_assignments?user.id=bb8d1e9116254e08849307aa3450df44&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:10.767831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:10.826690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 430/855] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:10.904102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db7899-d321-4f55-83e3-8ba0d1c3160a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:10.943357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:10.944321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:10.944748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:10.945195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:10.945909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:10.947438 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:10.957678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:10.958858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:10.960083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:10.962435 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 426/856] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:10.968986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.035646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:11.079765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b2d9-425b-41a6-931f-e172e2a3fab0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.114777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.115676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.115904 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.116153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.116788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:11.117090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:11.117690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.118457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:11.123602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 431/857] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:11.132507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.196289 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:11.243032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a61c2-d65d-486b-be22-7aa63bf63511 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.243693 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13c84f-f2e8-4d54-90b8-a61dfcb0e1f8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.273691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:11.284529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.285694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.285970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.286240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.289652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.290523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.291017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.291787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.292628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:11.293129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:11.295335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:11.297275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:11.307196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:11.308454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:11.309391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 427/859] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:11.310167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 432/859] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:11.315017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.438734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977c0e9-8704-47b8-857e-dd11b0e93e08 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.464877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:11.478624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.479275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.479601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.479878 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.480550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:11.480856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:11.482278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:11.491483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 428/860] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:11.496816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.606728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1240cda-8f8c-4e56-8271-7b0842d4ab71 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.641918 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.642560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.642841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.643120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.654343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:11.655899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:11.671490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:11.672638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:11.674003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:11.674003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:11.674003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:11.674003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 433/861] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:11.679870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.788184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4c69-f7b0-4a9c-9303-6eafbb8d19a6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.826919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.827570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.827853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.828131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:11.828774 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:11.829078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:11.830445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:11.843107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 429/862] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:11.849087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:11.892392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:11.954271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05f89-5743-4312-a7ae-7d096536f9b7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:11.990096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:11.990719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:11.990991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:11.991290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.000233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.002518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:12.016173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:12.017126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:12.018428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 434/863] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:17:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:12.023251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.073035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.132453 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e50ce-906b-48f6-a9a5-429f9daf68dd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:12.168199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:12.168808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.169085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.169357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.170078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:12.170383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.171837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603460-acd9-4ecd-86b5-b387647a3a87 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.189957 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 430/864] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:12.196436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.300425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.305116 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4cea5-904b-4bd5-b324-17bd9a70a863 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:12.343943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:12.344516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.344799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.345070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.424913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.428975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.439518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae94e-7eb8-4756-836a-714bc55d2fe1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.439997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae94e-7eb8-4756-836a-714bc55d2fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.439997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae94e-7eb8-4756-836a-714bc55d2fe1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.441554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 431/865] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:12.446633 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.446991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.447200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.465928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:12.467101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:12.530743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.535429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 435/866] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:12.540982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.546376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 432/867] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:12.642888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ea55-d2d1-4dfe-bb3c-6c3bf5439485 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:12.678483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:12.679496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.679890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.680255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.690665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.692731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:12.708023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:12.708930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:12.709805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:12.709805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:12.709805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:12.709805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 436/868] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:12.716194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.747885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.818855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184cc2-b4f9-40d7-8487-58adbbd64c22 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:12.824569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.856791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:12.857865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.858357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.858842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.859714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:12.860214 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.861969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.874604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 433/869] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:12.881145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:12.881732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.938678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43ec4-914f-413b-bfa1-93689a6dc08c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:12.975376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408a715-01df-433e-9503-48052e288695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:12.976258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a715-01df-433e-9503-48052e288695 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:12.976392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a715-01df-433e-9503-48052e288695 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:12.976622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a715-01df-433e-9503-48052e288695 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:12.977278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408a715-01df-433e-9503-48052e288695 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:12.978721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408a715-01df-433e-9503-48052e288695 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:12.987230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:12.997723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0b711-0790-434f-ba46-c53f41d332d6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.003234 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 437/870] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:12 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:17:13.020894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6dd5-0b91-4b39-ad48-aa9993f46f35 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.021196 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6dd5-0b91-4b39-ad48-aa9993f46f35 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.021442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6dd5-0b91-4b39-ad48-aa9993f46f35 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.023048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 438/871] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:13.029659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.030182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.030686 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.056299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:13.056507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.056507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.056507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.056806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:13.056901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.062166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.063861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:13.077816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:13.079394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:13.080593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:13.083337 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 434/872] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:13.085607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:13.137661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 439/873] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:13.204768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aa498-171d-4d5a-a864-824c240422aa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.241793 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.242406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.242683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.242956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.243666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:13.243980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.245390 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.258039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 435/874] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:13.263595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:13.307637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:13.369917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9e26-f028-4ecd-ba27-6e714a604286 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.405440 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.406060 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.406346 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.406622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.407389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=36aa8a9229e84ff29445df661f752fcb) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:13.407702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.409146 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.416122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 440/875] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:13 2022] GET /identity/v3/role_assignments?user.id=36aa8a9229e84ff29445df661f752fcb&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:13.434916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:13.450887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:13.565261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f62ed-83b5-4bd3-a79d-4f6a31007716 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.604729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.605372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.605658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.605940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.606535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.607945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2e9dfd42-f5ba-4d2b-b617-732027d1c341 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:13.617451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:13.618450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:13.619329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 436/876] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:13.626604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:13.636753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443af5-722a-488e-8731-858ec4117bdf None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.637034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443af5-722a-488e-8731-858ec4117bdf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.637298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443af5-722a-488e-8731-858ec4117bdf None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.638747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 437/877] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:13.643781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.644068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.644974 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.663430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:13.664290 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:13.752854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cc71a-74be-4a56-945a-87a310925e22 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.761368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 438/878] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:13.761957 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:13.795286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.795906 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.796189 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.796466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.797098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:13.797406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.798780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.804432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 441/879] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:13.809944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:13.814638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:13.914769 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae321816-e6e4-4d22-915e-89ad76f55499 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:13.953946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:13.954554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:13.954832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:13.955101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:13.955806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:13.956113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:13.957985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e492c8-54ce-49b0-b76b-2f36dbf75bb2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:13.962830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 439/880] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:14.031428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:14.161771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f6355-7f64-490d-bc5e-25d7fd7c6515 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:14.198698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:14.199688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:14.200203 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:14.200695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:14.201555 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:14.203120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c43f-f8e7-44fb-b3b8-de47635a4d8d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:14.221979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 442/881] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:14 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:15.381714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782e64-7fd6-4c47-ad58-572f7834af52 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:15.382723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782e64-7fd6-4c47-ad58-572f7834af52 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:15.382723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782e64-7fd6-4c47-ad58-572f7834af52 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:15.383877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 440/882] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:15.389211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a2323-1f78-4e70-8c9e-9e3e9545662f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:15.389617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a2323-1f78-4e70-8c9e-9e3e9545662f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:15.389766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a2323-1f78-4e70-8c9e-9e3e9545662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:15.408888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-434a2323-1f78-4e70-8c9e-9e3e9545662f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:15.410165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-434a2323-1f78-4e70-8c9e-9e3e9545662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:15.434696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:15.491255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 443/883] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:15.757157 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82782e64-7fd6-4c47-ad58-572f7834af52 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:15.823343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481df46-2cb6-4a51-b00f-67a51bda89c4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:15.823691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481df46-2cb6-4a51-b00f-67a51bda89c4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:15.823907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481df46-2cb6-4a51-b00f-67a51bda89c4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:15.825465 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 444/884] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:15.825775 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:15.831127 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:15.831474 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:15.831770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:15.854215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:15.855575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:15.875817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82782e64-7fd6-4c47-ad58-572f7834af52 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:15.916412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:15.917540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:15.918037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:15.918532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:15.919350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:15.921040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:15.933042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:15.934134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:15.935037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:15.943490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 445/885] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:15.943645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 441/886] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:15.943732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.028969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.044729 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be738fe-7114-4368-9b91-bebfa6b6a7bb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.082374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.082988 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.083294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.083567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.084197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:16.084549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.085855 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.091112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 446/887] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:16.096736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.205361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5f15c-16a0-4c87-94d5-445a588fa0bc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.238411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.244609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.245518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.245809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.246080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.246709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:16.247013 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.248448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6143820-03ac-4bdd-8a5e-3a07d36456df None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.250566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.256564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 442/888] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:16.257949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6391b-622d-4c53-9bb8-f5ce2de3731a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.258237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6391b-622d-4c53-9bb8-f5ce2de3731a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.258512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6391b-622d-4c53-9bb8-f5ce2de3731a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.259965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 443/889] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:16.262421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6391b-622d-4c53-9bb8-f5ce2de3731a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.363251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a411586-e268-4016-bd0e-5e2542381f2c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.381550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6391b-622d-4c53-9bb8-f5ce2de3731a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.407420 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.408237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.408515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.408785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.417738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.418344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.418621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.418899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.419609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] RBAC: Adding query filter params (interface=public, service_id=40269e1b0f0f4282a5c5dd55acefe4e1, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:16.419725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:16.419965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.421314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a0f01-bc5c-4cc0-b348-bf563b9d105e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3c779c7b-1d7d-4c79-8c75-298195ebb0e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:16.424437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:16.425429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 447/890] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:16.428252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.429018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.429018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.434499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 444/891] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/endpoints?service_id=40269e1b0f0f4282a5c5dd55acefe4e1&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:16.435942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152daca-040b-4e17-ad78-9e5d209decf2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.436226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152daca-040b-4e17-ad78-9e5d209decf2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.436500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152daca-040b-4e17-ad78-9e5d209decf2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.437671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 445/892] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:16.438414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152daca-040b-4e17-ad78-9e5d209decf2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.451617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:16.452820 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:16.456154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.497751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.521020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 448/893] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:16.521860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.543474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152daca-040b-4e17-ad78-9e5d209decf2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.582716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.583454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.583729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.584016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.584657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:16.584965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.586561 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ed43-e704-4924-89fd-256c61e5f9aa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.601225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 446/894] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:16.601225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.601225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.601225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.616003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:16.617125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:16.626357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15965d2d-d0bd-4f8c-8cce-1392e5271821 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.662049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.662658 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.662942 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.663237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.663863 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.665226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.670874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 449/895] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:17:16.671505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.675552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.690499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 447/896] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:16.791894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659f4c-2bac-49de-ab10-daa14015aac9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.807986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.830918 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.832134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.832411 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.832681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.848332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.850617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:16.867248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:16.868207 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:16.869094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:16.869094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:16.869094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:16.869274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.873215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 450/897] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:16.874454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:16.928449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8a17-6556-41c9-9933-12a795ca4e04 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:16.939835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:16.977941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:16.978556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:16.978880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:16.979169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:16.979866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:16.981224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:16.993025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df6339-9cbb-4297-af56-8b9255a530dc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.003264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 448/898] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:16 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:17:17.003264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.053194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.054111 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.055262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.055696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.056519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:17.057514 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.059682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.078605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 451/899] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:17.085602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.145556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241b15-f52a-44fd-8206-450a14911419 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.162581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.200667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.201816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.202243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.202654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.203472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.203941 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147c39-16c0-40c7-8146-d707e7d4f9b8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.205160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:17.214948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:17.215942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:17.217497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 449/900] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 03:17:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:17.221745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.223431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.253890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.254687 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.254972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.255894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.265669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.267131 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:17.282338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:17.283387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:17.284362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:17.287330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 452/901] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:17.291019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.339325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d11bc-6cca-43d4-a97a-bfd0dc525f54 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.387840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.388972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.389585 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.390086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.391271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:17.392411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.394361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.402038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 450/902] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:17.408416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.432015 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1c35-3688-4af3-a9c6-fb57054b09d1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.466143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.468161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.468996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.469453 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.469827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.470733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:17.471261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.473368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.503030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 453/903] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:17.504738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.543076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46b95c-f573-4984-bff3-a8d569894c36 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.545843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.591529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.593320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.593747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.594182 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.594964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:17.595441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.596950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.605160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 451/904] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:17.611048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.630778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286162f-73df-48b3-bcad-4e8d0c0783a2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.673734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.674267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.674542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.674811 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.739700 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f734b-3f6e-4dc1-bad4-c4da81458e48 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.741704 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.759549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.761573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.785631 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.786264 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.786778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.787058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.787787 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] RBAC: Adding query filter params (interface=public, service_id=5dc867a480af46508d086b9a3934ccc0, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:17.788095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.790119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.795486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:17.802988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 452/905] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/endpoints?service_id=5dc867a480af46508d086b9a3934ccc0&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:17.804279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.891228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 454/906] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:17:17.895468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:17.914366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4387c-0601-49ba-89c4-cedf7b26a4c7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:17.959865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:17.960545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:17.960826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:17.961117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:17.961721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:17.963483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:17.979185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 453/907] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 03:17:18.016433 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad018c-3372-499b-a1f0-b060f8eb2e3b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:18.049439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:18.056563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:18.057210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.057493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.057768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.069833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:18.071350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:18.088247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:18.089260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:18.090165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:18.090165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:18.090165 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:18.094068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 455/908] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:18.095486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:18.128999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:18.203392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297301ce-ffff-4c3b-bb2f-2dfce33e1165 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:18.238219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:18.238749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.239026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.239325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.239963 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:18.240265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:18.241613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:18.257621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 454/909] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:18.258945 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:18.365757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb96d5d-8bec-40bd-b7cc-c06364ec3fc0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:18.402204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:18.402830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.403108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.403413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.410419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:18.411891 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:18.423217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:18.424072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:18.425042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:18.429300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 456/910] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:18.431275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:18.524304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:18.534018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec337611-65c5-4e68-98c6-5def808a7728 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:18.569260 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:18.569877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.570153 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.570486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.571200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:18.571508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:18.572977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc92764-31d9-49b6-946f-45e3c02958f7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:18.590578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 455/911] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:18.592250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:18.637719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:18.700054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f42c-4661-431d-a4f5-e229f9e6914a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:18.734627 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:18.735540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.735974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.736387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.737246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] RBAC: Adding query filter params (role.id=6ee1bf77ba614b2c88de15ad3bd3d556, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=36aa8a9229e84ff29445df661f752fcb) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:18.737739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:18.739403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d79c3a-c50f-4477-9025-5fe84101fc79 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:18.749560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 457/912] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:18 2022] GET /identity/v3/role_assignments?user.id=36aa8a9229e84ff29445df661f752fcb&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=6ee1bf77ba614b2c88de15ad3bd3d556 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:18.951206 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da91b0-32d6-4e2a-9064-771c28f228b3 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.951498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da91b0-32d6-4e2a-9064-771c28f228b3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.951782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da91b0-32d6-4e2a-9064-771c28f228b3 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.953341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 456/913] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:18.959405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cd45d-2a00-4a32-ae43-8ca72baf5d46 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:18.959698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cd45d-2a00-4a32-ae43-8ca72baf5d46 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:18.959983 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cd45d-2a00-4a32-ae43-8ca72baf5d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:18.978142 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-714cd45d-2a00-4a32-ae43-8ca72baf5d46 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:18.979368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-714cd45d-2a00-4a32-ae43-8ca72baf5d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:19.047658 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 458/914] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:19.278221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d736-e014-4109-84b3-e76d554d4f5c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.278516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d736-e014-4109-84b3-e76d554d4f5c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.278698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d736-e014-4109-84b3-e76d554d4f5c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.280170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 457/915] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:19.285754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.286292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.286776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.306099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:19.307666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:19.326196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d736-e014-4109-84b3-e76d554d4f5c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.341637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:19.391173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 459/916] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:19.438195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d736-e014-4109-84b3-e76d554d4f5c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.473742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:19.474375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.474657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.474933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.475604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:19.477016 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:19.486262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:19.487300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:19.488172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 458/917] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:19.493355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.594735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d86e5-daac-4b8e-b8b6-1d8183ae8d43 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.629645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:19.630259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.630594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.630874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.631565 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:19.631881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:19.633254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:19.638259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 460/918] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:19.644109 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.671840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:19.673271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.712481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:19.762281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039d38a-0451-4923-944b-d4a2677c360f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.797829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfd693-18b5-477e-bb06-d8d5dfa4d82b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.800042 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:19.801564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.802050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.802831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.803709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:19.804237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:19.805863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:19.811845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 459/919] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 03:17:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:19.819762 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.841854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:19.842554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.842835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.843133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.843858 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:19.845396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:19.855170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:19.856185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:19.856993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:19.861648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 461/920] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:19.862610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:19.919381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:19.950479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cbbd2-1db0-415e-a42d-a84e1f584531 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.974084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:19.981650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930a100-a366-46e9-a1bc-ac28a09e6f55 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:19.994794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:19.995446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:19.995732 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:19.996010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:19.996752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:19.998121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.019405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.020226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.020569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.020943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.021685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:20.022050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.023023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 460/921] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:20.023648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.045483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 462/922] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:20.048150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:20.182044 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077b16a-54bd-4f80-89ea-b2056bd09855 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:20.214082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:20.225139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.225795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.226049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.226311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.226956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:20.228275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.229673 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9335-4eb2-49e5-8ea4-510ac8a7499f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.235422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 461/923] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:20.242704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:20.264581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18fba-92d4-425f-bcf1-98085b0e391e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.265205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18fba-92d4-425f-bcf1-98085b0e391e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.265563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18fba-92d4-425f-bcf1-98085b0e391e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.267053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 462/924] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:20.272528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.272808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.273082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.296482 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:20.297677 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:20.369344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 463/925] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:20.370651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1160-41ca-438f-b715-ff168b431ed8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:20.415842 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.416714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.417152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.417573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.418419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] RBAC: Adding query filter params (interface=public, service_id=4253b2d059c74e70b2320dec3140a5c7, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:20.418880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.420424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.426453 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 463/926] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/endpoints?service_id=4253b2d059c74e70b2320dec3140a5c7&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:20.435237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:20.538194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837f0a-4ed6-4add-bb8d-b1bfcf080375 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:20.563359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:20.574795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.575498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.575981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.576258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.576868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.578239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.585497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 464/927] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:17:20.640247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:20.669122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:20.755980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5ec7-cb40-403c-b6b9-924fac131dfc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:20.790705 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.791364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.791640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.791914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.792498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.793861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:20.803646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:20.804560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:20.806070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 464/928] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:20.810763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:20.920862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c36f9a-d51e-4ef9-acaa-3a9484ecea60 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:20.957457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:20.958445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:20.958960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:20.959474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:20.960370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:20.960883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:20.962452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c545f-c4ee-41d7-82c7-c556f75e7329 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:20.968008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 465/929] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:20.976742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:21.017657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56af21-d86f-49a7-8f5c-d0ad18e35ce7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.017865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56af21-d86f-49a7-8f5c-d0ad18e35ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.018148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56af21-d86f-49a7-8f5c-d0ad18e35ce7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.019896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 466/930] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:21.028578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.028997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.029480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.050509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:21.051961 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:21.070211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:21.087526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58beaeb2-1193-4e20-b8d6-21cd8d4a6933 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:21.107125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:21.127731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:21.128910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.129186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.129454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.130073 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:21.130373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:21.131835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:21.139463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 467/932] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:21.139614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 465/932] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:21.145221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:21.248287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b0312-6496-4596-8e5e-8ed919293363 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:21.283853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:21.284841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.285319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.285773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.286633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:21.288227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:21.312026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 466/933] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:17:21.416367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:21.530426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125fc90-1d84-41d0-b272-f3fb540b584c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:21.574847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d6417-9964-4782-8bde-472e61744357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:21.575832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6417-9964-4782-8bde-472e61744357 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.576325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6417-9964-4782-8bde-472e61744357 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.576793 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6417-9964-4782-8bde-472e61744357 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.588114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d6417-9964-4782-8bde-472e61744357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-619d6417-9964-4782-8bde-472e61744357 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:21.593405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:21.594317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:21.595175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:21.597547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 468/934] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:21.603341 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:21.612582 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:21.709367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceeeff-bcd1-46a6-864e-2d308bec0541 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:21.744199 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:21.744811 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.745094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.745363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.746026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:21.746328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:21.747741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:21.756823 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 467/935] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:21.762445 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d6417-9964-4782-8bde-472e61744357 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:21.806078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:21.872137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d6417-9964-4782-8bde-472e61744357 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:21.908870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:21.910409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:21.910895 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:21.911400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:21.922636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:21.924567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:21.940410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:21.942014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:21.942907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:21.942907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:21.942907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:21.945154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 469/936] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:21.955192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.024551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:22.061480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd311c-2349-4eee-871b-fbb92ed39f4f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.098011 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.098888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.099338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.099754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.100533 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:22.100979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.102481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.116094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 468/937] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:22.121840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.228789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9beb29-1911-44ec-89cb-93f1d33a34ba None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.231449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:22.273451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.274114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.274393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.274665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.283567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.285139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-45cff23b-f566-48fb-851d-cfe01b9eb7e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:22.296916 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:22.297839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:22.298756 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 470/938] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:17:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:22.306331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.370479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987254-fbc8-47a1-ade4-1ef2785de21b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.370814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987254-fbc8-47a1-ade4-1ef2785de21b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.371035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4987254-fbc8-47a1-ade4-1ef2785de21b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.386888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 471/939] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:22.388651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.388941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.389219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.407847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:22.409134 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:22.413284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d45d-0fd3-4998-8b10-b3dd97f9e277 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.441943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:22.453887 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.454552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.454834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.455106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.458298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:22.458638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.460251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.484702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 469/940] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:22.485698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.499282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 472/941] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:22.595897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6dd43-2e01-4c04-9176-c3b2e01ecf40 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.631485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.632112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.632389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.632710 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.642959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.644592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:22.659929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:22.660907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:22.661767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:22.661767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:22.661767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.661767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 470/942] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:22.668427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.751500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:22.773341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfef4cf-460b-4708-9ac1-108c86da152a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.800738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.811401 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.812025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.812305 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.812575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.813216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:22.813518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.814873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.826624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 473/943] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:22.833482 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:22.848068 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:22.915503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbdd9a-7cea-48d3-b17f-db9ce36f71f3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.945731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f24a8-9bb2-48f9-9df7-4a319569db0d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:22.951026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.951668 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.951948 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.952224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.953177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.954293 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:22.957476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING keystone.server.flask.common [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 03:17:22.975792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 03:17:22.975792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 03:17:22.975792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [parameters: {'id': '166c9944598d4b09b9dc3fc10fbfa1fd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 03:17:22.975792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 03:17:22.983800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:22.984437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:22.984717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:22.984984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:22.991966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5195f1df-942b-466a-83ce-3a6d13041ecd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 166c9944598d4b09b9dc3fc10fbfa1fd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 166c9944598d4b09b9dc3fc10fbfa1fd. Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:17:22.992405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 03:17:22.993201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 04 03:17:22.994129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 03:17:22.995080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 03:17:22.996162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:17:22.996929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'id': '166c9944598d4b09b9dc3fc10fbfa1fd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.997775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:22.998545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 166c9944598d4b09b9dc3fc10fbfa1fd. Oct 04 03:17:23.001205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:23.003283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.003448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 471/944] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 4 03:17:22 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:23.003448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0c88-b9ce-4f29-8ca3-7d7d71e379f6 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.003448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0c88-b9ce-4f29-8ca3-7d7d71e379f6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.003448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0c88-b9ce-4f29-8ca3-7d7d71e379f6 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.007176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 472/945] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:23.007176 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba0c88-b9ce-4f29-8ca3-7d7d71e379f6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.007683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8408038d-c9d6-4b1d-80fb-2a4dafce2048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:23.014098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:23.015130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:23.016292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:23.023432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 474/946] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:23.023432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.023432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.023432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.038033 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:23.039344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:23.114728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 475/947] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:23.115594 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.133133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.142643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba0c88-b9ce-4f29-8ca3-7d7d71e379f6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.186630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.187601 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.187888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.188162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.198602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.200246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:23.214464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:23.215576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:23.216467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:23.219290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 473/948] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:23.221421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.240223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdf560-ea4a-4a62-a848-2e6272a22a91 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.272222 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.278930 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.279579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.279865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.280140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.280831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:23.281139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.282680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.295604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 476/949] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:23.301709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.362137 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e269eeb-d635-4867-9e28-2e6ccce8e534 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.400094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.400971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.401451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.401925 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.402779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:23.403320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.404979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.422721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc4572-e5cd-447c-998d-d8aafa9d6fbf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.423712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 474/950] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:23.437530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.457701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.477739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.479131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.479665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.480151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/36aa8a9229e84ff29445df661f752fcb/roles/6ee1bf77ba614b2c88de15ad3bd3d556` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.484313 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=36aa8a9229e84ff29445df661f752fcb, role_id=6ee1bf77ba614b2c88de15ad3bd3d556)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.486293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b99b9-2062-4862-ac09-b380afd4f5b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.508905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 477/951] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:23 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/36aa8a9229e84ff29445df661f752fcb/roles/6ee1bf77ba614b2c88de15ad3bd3d556 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:17:23.583087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff369ba-f4b5-4c9c-8378-4ebfc198236e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.606897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e4065-d8b0-4381-b84b-8f675cf93ec2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.607276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e4065-d8b0-4381-b84b-8f675cf93ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.607494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e4065-d8b0-4381-b84b-8f675cf93ec2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.609194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 478/952] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:23.615771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.616092 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.616341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.629878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.631380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.631666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.631941 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.632932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.634342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.637222 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:23.638458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:23.646039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:23.647304 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:23.648216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:23.651248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 475/953] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:23.654722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.706107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.732677 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 479/954] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:23.753556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:23.777628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa56f7-93cc-46e4-9fd9-9f4e141a9026 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.812092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.812704 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.812964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.813243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.813875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:23.814183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.815626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.820931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 476/955] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:23.828431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:23.929656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e90e97-7c88-49bf-80d3-ff22393d5320 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:23.964524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:23.965894 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:23.966399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:23.966874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:23.967737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:23.968316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:23.969875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:23.980478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 480/956] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:23.982161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:24.017127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:24.077997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:24.093594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9588-806b-4e28-80f3-dd726cda7d11 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:24.129819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:24.130434 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:24.130712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:24.130984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:24.131679 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:24.133042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:24.150136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:24.155505 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d973f5c-e977-482d-9ab8-219d837b6252 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:24.157803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 477/957] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:24.206117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:24.207267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:24.207580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:24.207820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:24.208397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:24.209839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:24.219365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:24.220508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:24.222048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 481/958] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:24.226898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:24.342982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30faa120-a62e-4f3a-9625-b67e2a754ca9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:24.390721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:24.391751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:24.391751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:24.391942 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:24.392539 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:24.392838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:24.394742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cd24d-83f3-4743-bc2c-eab3c1b694fe None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:24.399966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 478/959] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:24.405309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:24.513207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ecfd-48dd-448c-8604-c305c70f398d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:24.516941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:24.553334 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:24.553949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:24.554226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:24.554496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:24.555119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:24.555446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:24.556788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819576d-cc4a-45ad-9f09-e6d71f00e16c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:24.561708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 482/960] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:25.740385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470254ef-ba2d-4887-ae5b-86e68d289775 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:25.740900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470254ef-ba2d-4887-ae5b-86e68d289775 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:25.740900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470254ef-ba2d-4887-ae5b-86e68d289775 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:25.742432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 479/961] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:25.747664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:25.748072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:25.748240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:25.767067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:25.768283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:25.803035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691cc62-55fa-48b8-ab9b-95337c3447bd None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:25.803344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691cc62-55fa-48b8-ab9b-95337c3447bd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:25.803628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691cc62-55fa-48b8-ab9b-95337c3447bd None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:25.805168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 480/962] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:25.810249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:25.810536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:25.810817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:25.831662 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:25.833480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:25.845004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 483/963] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:25.897566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:25.909014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 481/964] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:26.112306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.179886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.210574 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:26.214945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954fc54-ee84-47b7-b168-5ed035f7dbdc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.252840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.253564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.253945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.254224 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.265454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:26.270658 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:26.271626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:26.272491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 484/965] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:26.277911 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.308389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8a744-852b-43da-a4f4-e2ac94ce9232 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.345035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.345640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.345917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.346187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.347397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.348895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.364366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 03:17:26.364366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 03:17:26.364366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [parameters: {'user_id': '23ef4b120ee44aacbcdaa146ff93762c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 03:17:26.364366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 04 03:17:26.383242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:17:26.384320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 03:17:26.385357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 03:17:26.386230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 03:17:26.387091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 03:17:26.388157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'user_id': '23ef4b120ee44aacbcdaa146ff93762c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:26.389098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 03:17:26.389872 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 03:17:26.390746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.392826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 482/966] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 03:17:26 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:26.398869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.414011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba823-b920-43ae-ad6e-1eb8286d0674 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.455926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.456927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.457392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.457847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.458770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:26.459388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.461178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.462521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:26.474382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 485/967] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:26.483211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.505962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:26.515695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5ee29-2a0f-47a3-a1f9-63b0b08bc767 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.557133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.558566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.559001 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.559635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.567921 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.569556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a5bb47ff-1888-4575-9f33-ed0155f7993d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:26.582243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:26.583287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:26.584264 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 483/968] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:26.584870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb095-5f2f-4918-a615-27302ec24170 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.584870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb095-5f2f-4918-a615-27302ec24170 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.585502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb095-5f2f-4918-a615-27302ec24170 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.589112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 484/969] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:26.589833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feb095-5f2f-4918-a615-27302ec24170 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.608021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375ce46-0af8-4d1c-b4d3-7dd76a358d48 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.646612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.647234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.647523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.647800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.660992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.662460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-102e732a-2e59-4590-8a64-0de209c93bc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:26.677927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:26.678965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:26.679881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:26.679881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:26.679881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:26.679881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 486/970] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:26.680536 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.680536 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.680768 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.700666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:26.702592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:26.706038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feb095-5f2f-4918-a615-27302ec24170 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.720228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:26.742171 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.742810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.743055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.743358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.744124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:26.745145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.745861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b76be-ba1e-4afa-9c6e-d45b8660dd97 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.776494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 487/971] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:26.777373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:26.780368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:26.781767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 485/972] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:26.782631 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb6e7-287e-4a8a-913d-d25c627b2e32 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.783040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb6e7-287e-4a8a-913d-d25c627b2e32 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.783246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb6e7-287e-4a8a-913d-d25c627b2e32 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.787305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 486/973] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:26.790366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.790662 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.790944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.811976 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:26.813475 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:26.898263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 487/974] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:26.911088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838531e5-86ec-4326-a388-02e4c2abf25a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:26.955344 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:26.956394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:26.956883 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:26.957362 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:26.958216 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:26.958737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:26.960606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:26.974421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 488/975] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:26.979737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.044244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.100790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c0b16-a20f-411a-a2b2-77a7a5bb3127 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.102804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.121406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.146491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.147449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.147732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.148008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.155682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.157076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.162113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5a0a-0e2c-424b-b8e1-4200ef23b10f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:27.171128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:27.172287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 03:17:27.173361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:27.174468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 488/976] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:17:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:27.179505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.199013 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.199710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.199996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.200274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.200862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.202226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:27.211619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:27.212864 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:27.213867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 489/977] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:27.214833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.284366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14367aa4-3c60-444d-8ec9-f95f51ebc499 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.314989 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae564bf-e5bf-4c41-8c71-4990518448d0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.321483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.322092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.322375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.322651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.323330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.324705 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.340074 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 489/978] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:27 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:17:27.340995 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.342030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.360129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.360760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.361043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.361324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.362073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:27.362928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.364447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.382480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.392844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 490/979] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:27.398760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.472841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2bb64-e40a-43d5-877d-4903d58053f6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.509709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68e68-ffa3-4afd-9906-96d100ba4c1d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.513505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.514134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.514416 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.514688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.515351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:27.515674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.517072 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.523201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 490/980] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:27.528479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.549558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.550714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.550997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.551356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.604992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.627178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.628708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.630534 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.645310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d277098-6c3e-4c16-9831-bc829ff9eaf4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.681160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.681864 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.682216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.682568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.683344 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:27.683776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.685245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.690964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 491/981] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:27.696552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.741634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 491/982] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:17:27.742601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.807518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3c7d-ef56-4814-8c7a-ac3ad95d5bbd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.829955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.851444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.852061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.852343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.852619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.852894 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d11e07-c718-4046-83d0-164031ab9540 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:27.853331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] RBAC: Adding query filter params (interface=admin, service_id=4253b2d059c74e70b2320dec3140a5c7, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:27.853642 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.855021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.860371 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 492/983] 10.222.0.15 () {56 vars in 1249 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/endpoints?service_id=4253b2d059c74e70b2320dec3140a5c7&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:27.865842 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.882846 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:27.896818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:27.897466 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:27.897749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:27.898031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:27.908439 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:27.910525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:27.931927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:27.933035 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:27.934142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:27.934142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:27.934142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:27.939778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 492/984] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:27.940652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:27.980112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd0b98-676b-4b57-8d3c-ca2b9062daea None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:28.015414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:28.016088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:28.016533 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:28.016916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:28.017621 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:28.019673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:28.026554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 493/985] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 03:17:28.048787 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7aff-9b18-4c34-975b-d94c915faad9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:28.084993 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:28.092114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:28.092746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:28.093026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:28.093299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:28.093950 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:28.094258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:28.095694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:28.114282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 493/986] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:28.115607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:28.199959 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:28.225154 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d097c8-cbf1-48bc-891c-01655fd9ce20 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:28.260868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:28.261681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:28.262051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:28.262419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:28.270263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:28.271790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:28.282932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:28.284008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:28.285290 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:28.286659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 494/987] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:28.292261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:28.396782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feaee5-b711-43d1-8cf0-bf2fc66eecaa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:28.431963 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:28.432914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:28.433413 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:28.433892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:28.434779 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:28.435319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:28.436962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf01d2-0454-4f51-9c53-615cb7f4e67b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:28.449568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 494/988] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:28.455132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:28.558818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3455-761a-48bf-94df-49f284bb5caf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:28.594174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:28.594783 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:28.595061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:28.595360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:28.596081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] RBAC: Adding query filter params (role.id=6ee1bf77ba614b2c88de15ad3bd3d556, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=36aa8a9229e84ff29445df661f752fcb) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:28.596391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:28.598443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a8332-91f4-4077-829d-eb43e51130e6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:28.611397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 495/989] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:28 2022] GET /identity/v3/role_assignments?user.id=36aa8a9229e84ff29445df661f752fcb&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=6ee1bf77ba614b2c88de15ad3bd3d556 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:29.527248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76272-d545-4a34-95cb-f849a7ed7c61 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:29.527754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76272-d545-4a34-95cb-f849a7ed7c61 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:29.527859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76272-d545-4a34-95cb-f849a7ed7c61 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:29.529114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 495/990] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:29.534604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:29.534973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:29.535170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:29.541574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:29.555007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:29.556202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:29.625859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 496/991] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:29.891962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac76272-d545-4a34-95cb-f849a7ed7c61 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:29.948872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:29.997584 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac76272-d545-4a34-95cb-f849a7ed7c61 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.031518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.032140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.032416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.032684 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.033263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.034695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9ce6da2c-e1f8-4612-89a8-1dfed0e7d2d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:30.043325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:30.044284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:30.045115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.047461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 496/992] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:30.050339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.156667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61a20-5d09-4e6e-8078-6f7fb27bd481 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.185904 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:30.191750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053267eb-0a0e-4d6c-b7d2-94fc6d485da1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.192632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053267eb-0a0e-4d6c-b7d2-94fc6d485da1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.192738 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.193024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053267eb-0a0e-4d6c-b7d2-94fc6d485da1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.193297 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.193579 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.193856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.194486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:30.194582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 497/993] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:30.194793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.196273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.199479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.199789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.200051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.206628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 497/994] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:30.207976 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.218710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:30.219978 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:30.294969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 498/995] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:30.312076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182eecd-794b-4c4d-966a-b4c6eae6f5f5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.347576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.348184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.348463 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.348737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.349357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:30.349661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.351025 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.360812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 498/996] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:30.362077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.416107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:30.470148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321173f5-e49b-4c12-bc79-dc73251126cc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.502968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.503596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.503873 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.504143 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.504840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] RBAC: Adding query filter params (interface=public, service_id=952bf35d667943b1888edf0c0eb36485, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:30.505146 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.506491 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.511749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 499/997] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/endpoints?service_id=952bf35d667943b1888edf0c0eb36485&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:30.517092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.566931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.579507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:30.627161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62967e06-bd22-4788-a1e8-4739ef35a726 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.662947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.663610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.663888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.664159 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.664777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.666249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.673510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 499/998] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:17:30.692365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1a845-9e13-4393-b55a-7c411cfe240d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.707003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:30.733748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.734671 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.735125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.735576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.736350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.737965 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-cdd42e94-a0a5-41e3-bcfb-cac92e7edf09 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:30.748177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:30.749459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 03:17:30.750497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:30.758330 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 500/999] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:30.759849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:30.791391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382112-44ad-4094-88c7-61548c1c481c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.791721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382112-44ad-4094-88c7-61548c1c481c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.792123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382112-44ad-4094-88c7-61548c1c481c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.793728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 501/1000] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:30.800244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.800581 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.800790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.821589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:30.823790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:30.873690 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd816b9-ae22-4431-8aeb-1c2bc2737881 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:30.900559 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:30.902919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 502/1001] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:30.917329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:30.917875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:30.918150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:30.918424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:30.919053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:30.919383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:30.920739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:30.930413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 500/1002] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:30.931657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:31.028709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbb86b-6cb0-40be-9f6b-b4330664349d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:31.031073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:31.066175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:31.066784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:31.067230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:31.067667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:31.068295 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:31.068598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:31.069948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:31.074962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 503/1003] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 03:17:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:31.080808 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:31.172948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:31.186772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e8f94-febf-4b55-8ecb-fcd1c498395c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:31.224490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:31.225102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:31.225382 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:31.225657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:31.226326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:31.227725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:31.244105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 501/1004] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:31.280492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8ffe-dc7c-462a-9afe-5a56fb4d1c1a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:31.324304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:31.324947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:31.325244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:31.325487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] PATH_INFO: `/v3/services/image` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:31.326072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:31.327568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:31.337601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:31.338697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:31.344952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 504/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 03:17:31 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:31.345753 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:31.405266 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:31.465721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b77e2-2b5b-492a-810b-3d91ba3f1367 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:31.499205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:31.499823 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:31.500104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:31.500376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:31.501003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] RBAC: Adding query filter params (name=image) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:31.501309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:31.502675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c97e2-114f-421b-a4c7-d16dade18080 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:31.508092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 502/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:31.513805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:31.543185 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:31.618094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605257-d345-4ee6-ad4b-a9894cfae922 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:31.651937 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:31.652548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:31.652828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:31.653101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:31.653726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] RBAC: Adding query filter params (type=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:31.654032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:31.655452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbb40-e33e-43e5-83d2-3611767e449b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:31.667243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 505/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:32.852660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05af53-b2a7-49f9-b642-a0a9139cc172 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:32.853168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05af53-b2a7-49f9-b642-a0a9139cc172 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:32.853271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05af53-b2a7-49f9-b642-a0a9139cc172 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:32.854695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 503/1008] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:32.859770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:32.860187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:32.860353 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:32.878776 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:32.879999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:32.945726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 506/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:33.217056 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05af53-b2a7-49f9-b642-a0a9139cc172 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.323351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05af53-b2a7-49f9-b642-a0a9139cc172 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:33.338584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:33.360501 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:33.361187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.361477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.361751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.362357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:33.363751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5e13c7d9-33f5-4d5b-81c7-949e891b11ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:33.374466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:33.375546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 03:17:33.376398 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:33.381451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 504/1010] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:33.386910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.410487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:33.426210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b2cb-b27e-4dae-9037-fdb566475798 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.426494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b2cb-b27e-4dae-9037-fdb566475798 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.426775 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b2cb-b27e-4dae-9037-fdb566475798 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.428725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 505/1011] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:33.433381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.434324 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.434609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.456842 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:33.458193 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:33.496912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27b895-2919-4456-be4e-badcd4cf8d63 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:33.529576 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 506/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:33.538499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:33.539121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.539451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.539731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.540418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:33.540734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:33.542164 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:33.547575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 507/1013] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:33.553164 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.655210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193cda9-cd68-4959-a70f-b29cb685cb59 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:33.663269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:33.692076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:33.692718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.693002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.693278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.693931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:33.694243 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:33.695716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:33.706308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 507/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 03:17:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:33.708058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.797538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:33.802892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.816266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebeb72-ed91-4f48-975e-9b42a5a3a2e5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:33.850266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:33.850866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.851160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.851448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.852135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:33.853486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf6bdc-1f92-4610-a947-60b9569fe090 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:33.873852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 508/1015] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:33.875075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1be4-af97-4e83-89db-e894253ce1f2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.875394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1be4-af97-4e83-89db-e894253ce1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.875674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1be4-af97-4e83-89db-e894253ce1f2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.877088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 509/1016] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:33.882750 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.883030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.883345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.904256 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:33.905483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:33.915404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13a44a-c664-4986-ad5d-f03f2c2d1a63 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:33.965374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d362-f407-4496-b537-613b1f846801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:33.966050 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d362-f407-4496-b537-613b1f846801 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:33.966305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d362-f407-4496-b537-613b1f846801 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:33.966580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d362-f407-4496-b537-613b1f846801 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:33.967794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d362-f407-4496-b537-613b1f846801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:33.969320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d362-f407-4496-b537-613b1f846801 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:33.971113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 508/1017] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:33.977449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d362-f407-4496-b537-613b1f846801 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:33.982602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 510/1018] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:33 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:34.013847 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:34.089381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d362-f407-4496-b537-613b1f846801 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:34.125150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:34.125899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:34.126278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:34.126652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:34.127470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:34.129198 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6de2df-d22c-4024-b707-99dd7147ecef None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:34.157707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 509/1019] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:33 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:17:34.251599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:34.327837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:34.359539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431f36c-6bb2-4cc0-a876-c314618075bf None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:34.397998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:34.398613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:34.398886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:34.399181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:34.399836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:34.401196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78f2f3-2dfe-43c0-96a5-8b9877e38437 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:34.420762 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 511/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:17:34 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:36.189692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ea7d-587f-45bc-b6b2-e978f36675ca None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.193670 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ea7d-587f-45bc-b6b2-e978f36675ca None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.193670 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ea7d-587f-45bc-b6b2-e978f36675ca None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.193670 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 510/1021] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:36.197533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171c62-1bc2-47aa-a5fa-a644f2c50e31 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.197988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171c62-1bc2-47aa-a5fa-a644f2c50e31 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.198075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171c62-1bc2-47aa-a5fa-a644f2c50e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.217031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6e171c62-1bc2-47aa-a5fa-a644f2c50e31 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:36.218263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6e171c62-1bc2-47aa-a5fa-a644f2c50e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:36.298091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 512/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:36.344082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79d060-86af-4ca2-8803-7817827dbc87 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.344392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79d060-86af-4ca2-8803-7817827dbc87 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.344641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79d060-86af-4ca2-8803-7817827dbc87 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.346099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 511/1023] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:36.351319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabfa5d-f013-47e7-b05f-209c1ccdf5d8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.351609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabfa5d-f013-47e7-b05f-209c1ccdf5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.351884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabfa5d-f013-47e7-b05f-209c1ccdf5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.370080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5fabfa5d-f013-47e7-b05f-209c1ccdf5d8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:36.371279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5fabfa5d-f013-47e7-b05f-209c1ccdf5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:36.429588 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:36.445097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 513/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:36.567509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79d060-86af-4ca2-8803-7817827dbc87 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:36.601051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed73a6-c029-4fd4-b141-c70bb53ef949 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.601351 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed73a6-c029-4fd4-b141-c70bb53ef949 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.601604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed73a6-c029-4fd4-b141-c70bb53ef949 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.603205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 514/1025] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:36.608081 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.608575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.609043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.630173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:36.631392 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:36.662107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:36.681174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79d060-86af-4ca2-8803-7817827dbc87 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:36.706592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 515/1026] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:36.711094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:36.716965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:36.717602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.717881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.718153 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.729210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:36.734077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:36.735088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:36.736043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 512/1027] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:36.741429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:36.816135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce904b-5d33-4ae8-953a-4de36ca8fe1c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:36.833496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:36.851890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f9272-3dd3-488e-b686-ea872592accb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:36.852068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:36.852541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.852820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.853116 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.854213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:36.855682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:36.857414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 516/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:36.862765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:36.893042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:36.894215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:36.894497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:36.894972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:36.895468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:36.895782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:36.897146 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:36.907209 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 513/1029] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:36.912081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:36.923380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:36.971682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9b5ad-2f70-4a1f-8c11-283059013ad9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.006923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.007570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.007856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.008132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.008785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.010205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.014662 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c5d40-8c36-484b-8dbb-f86ab3ed6d31 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.030538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 517/1030] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:17:36 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:37.031398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.066717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.067408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.067748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.068296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.083975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.085894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.089181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:37.109373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:37.110638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:37.111935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:37.111935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:37.111935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:37.112521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 514/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:37.120414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.167512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21acf88-13d0-43bd-aab0-86be602c3d95 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.186278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.206525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d095-3844-448d-8a43-423800714752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.207223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d095-3844-448d-8a43-423800714752 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.207470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d095-3844-448d-8a43-423800714752 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.207743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d095-3844-448d-8a43-423800714752 None admin] PATH_INFO: `/v3/services/image` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.208336 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376d095-3844-448d-8a43-423800714752 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.209706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376d095-3844-448d-8a43-423800714752 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9376d095-3844-448d-8a43-423800714752 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:37.220233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:37.221559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:37.222566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 518/1032] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:37.229055 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d095-3844-448d-8a43-423800714752 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.240259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549b30-dda7-4ce7-84d0-0cb7a1c0c676 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.280832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.281570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.281949 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.282321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.282977 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:37.283321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.284782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.303768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 515/1033] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:37.309638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.343690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d095-3844-448d-8a43-423800714752 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.352011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.382863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.383523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.383854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.384142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.384778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] RBAC: Adding query filter params (name=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:37.385089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.386465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.391936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 519/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:37.397527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.420984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37960e64-7f28-4010-8a03-23bb879af9ff None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.442382 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.458047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.458730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.458965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.459270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.466617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.468126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:37.480587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:37.481611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:37.482586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 516/1035] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:37.487263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.511089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ec9a1-e813-489d-850d-28f37a2f925a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.551823 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.553098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.553654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.553939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.554577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] RBAC: Adding query filter params (type=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:37.554891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.556578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.562630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 520/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:37.568101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.597631 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf182-7c0a-42f9-97af-0e69c9048321 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.601915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.636235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.636876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.637138 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.637415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.638256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:37.638615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.640286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.659328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 517/1037] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:37.664924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.673847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46a3c8-14cd-4e74-8c5f-9bd1d242204a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.705912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.711513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.712127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.712409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.712685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.713389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] RBAC: Adding query filter params (interface=public, service_id=6dd2382097ad4a5ebb888221ca308c73, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:37.713696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.715060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.725571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 521/1038] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/endpoints?service_id=6dd2382097ad4a5ebb888221ca308c73&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:37.726599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:37.770570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f44d2-12f6-484c-9e46-15f98555d651 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.804995 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.805604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.805886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.806156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.829627 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.834296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef466f-49bf-430a-96d6-47f93916cc23 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:37.873975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:37.874871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:37.875194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:37.875452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.875556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:37.876082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:37.876952 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.877436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:37.887479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 522/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 03:17:37.924645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:37.997733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 518/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:17:37.999374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:38.106399 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79087fe-1e81-4f1c-863c-5ebddf107149 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:38.140355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:38.140993 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:38.141274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:38.141548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:38.152314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:38.153748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:38.168150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:38.169205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:38.170107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:38.170107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:38.170107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:38.170107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 523/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:38.176079 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:38.277469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91cd67-2641-4227-b909-edd91c7b54b5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:38.313747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:38.314366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:38.314648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:38.315476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:38.316150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:38.316470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:38.317841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:38.324958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:38.331446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 519/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:38.336846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:38.353842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:38.445021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb814-f78d-439f-afa8-0f2992615cc8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:38.479510 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:38.480348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:38.480795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:38.481238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:38.488504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:38.490099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:38.501420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:38.503085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:38.504245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:38.510861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 524/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:38.511960 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:38.614336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60968f02-e64b-43bd-acfb-20679be9529e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:38.649697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:38.650239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:38.650519 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:38.650795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:38.651518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:38.651826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:38.653247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1482a-d4a9-4356-aa94-004cf685c0be None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:38.665544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 520/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:38.671555 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:38.746832 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:38.775618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031479-544d-45c1-b346-b1dc0876e122 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:38.809591 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:38.810199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:38.810480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:38.810755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:38.811504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] RBAC: Adding query filter params (role.id=6ee1bf77ba614b2c88de15ad3bd3d556, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:38.811817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:38.813235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0820-eaa5-4e56-917f-57bc62f0f064 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:38.819359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 525/1045] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:38 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=6ee1bf77ba614b2c88de15ad3bd3d556 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:39.274460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8b2ff-cd06-422e-ab60-316ceea8b4e5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:39.274850 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8b2ff-cd06-422e-ab60-316ceea8b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:39.275043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8b2ff-cd06-422e-ab60-316ceea8b4e5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:39.276427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 521/1046] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:39.281572 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:39.281865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:39.282150 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:39.300773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:39.301966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:39.368706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 526/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:39.639650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8b2ff-cd06-422e-ab60-316ceea8b4e5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:39.702566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:39.746478 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8b2ff-cd06-422e-ab60-316ceea8b4e5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:39.780597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:39.781224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:39.781508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:39.781785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:39.792493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:39.797460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:39.798391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:39.799335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:39.803461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 522/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:39.804826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:39.871599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:39.906716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481401-ab15-4f75-81c7-9f93f9087a8a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:39.944146 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:39.944782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:39.945066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:39.945342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:39.946038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:39.946350 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:39.947801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb23506-8997-4045-9473-f62d0fcbf890 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:39.962383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 527/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:39.963716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.041289 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdab5c-2718-445c-ab7d-9b9e16837534 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.041573 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdab5c-2718-445c-ab7d-9b9e16837534 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.041847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdab5c-2718-445c-ab7d-9b9e16837534 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.043361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 528/1050] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:40.047954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.048240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.048515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.068417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2f52-60aa-43e6-82f1-6d1870ad9c45 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.068633 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:40.069811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:40.087665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.105479 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.106103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.106383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.106654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] PATH_INFO: `/v3/users/40a15daa71214f7bba26ca3794093473` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.120654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] RBAC: Authorizing `identity:get_user(user_id=40a15daa71214f7bba26ca3794093473)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.122084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.128556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 523/1051] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:39 2022] GET /identity/v3/users/40a15daa71214f7bba26ca3794093473 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:40.129272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.143659 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 529/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:40.228722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ce6c-30f6-47ca-8cdc-3e6b22e96392 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.262610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.263231 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.263512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.263780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.310419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.334261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.335741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.409718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.439663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 524/1053] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:17:40.445743 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.472691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.516283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355172d-87ab-4601-b065-716fb84cdcdf None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.539936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.553133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.553780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.554037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.554303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.555045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.556757 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.564570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867ce97-5799-419a-9345-1552394f8da2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:40.566091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:40.567038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:40.567896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 530/1054] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:40.572856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.604480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.605240 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.605524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.605843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.606953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.608538 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.615190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 525/1055] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:40.616119 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.678089 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5c9a-252c-497a-8a85-e63549a258d5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.702086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.714888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.715533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.715814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.716091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.716717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] RBAC: Adding query filter params (name=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:40.717024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.718385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.722462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45bdb-46ec-4532-9f71-826c919b1ece None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.723720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 531/1056] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:40.729193 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.762736 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.763374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.763648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.764030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.764747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f, user.id=40a15daa71214f7bba26ca3794093473) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:40.765056 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.766540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.772821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 526/1057] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/role_assignments?user.id=40a15daa71214f7bba26ca3794093473&scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:40.841278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f81d-b7e7-46e6-a16c-afc97aec9d59 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:40.881478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:40.882104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.882383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.882657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.883355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] RBAC: Adding query filter params (type=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:40.883664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:40.885049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c5ae-9e06-4b44-8653-052d55765a41 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:40.895713 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 532/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:40.896800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:40.909966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:40.964269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f79b-ebeb-4b40-a537-07dbefa9e3fa None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.964563 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f79b-ebeb-4b40-a537-07dbefa9e3fa None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.964845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f79b-ebeb-4b40-a537-07dbefa9e3fa None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.966319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 533/1059] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:40.971131 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:40.971526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:40.971804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:40.990184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:40.991405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:41.006917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb49a1-eaa2-45db-95c6-98d08fe25678 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:41.040472 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:41.041003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:41.041280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:41.041551 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:41.042020 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:41.042230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:41.043649 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:41.067631 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 527/1060] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:17:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:41.068326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 534/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:41.347456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:41.425080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:41.479178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2ecc6-214a-47de-b411-1d1c250ff6ce None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:41.514966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:41.515976 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:41.516466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:41.516973 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:41.532892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:41.539856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:41.540822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:41.541666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:41.549044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 528/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:41.550172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:41.673810 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7f47a-027f-4e32-9274-6e29bf4e8ad8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:41.711801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b6277-8c73-409f-ab37-14153273675c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:41.712423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b6277-8c73-409f-ab37-14153273675c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:41.712703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b6277-8c73-409f-ab37-14153273675c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:41.712975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b6277-8c73-409f-ab37-14153273675c None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:41.713636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b6277-8c73-409f-ab37-14153273675c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:41.713942 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b6277-8c73-409f-ab37-14153273675c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:41.715512 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b6277-8c73-409f-ab37-14153273675c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:41.728405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 535/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:41.732488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:41.840126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa3fb-2ee4-491b-a103-9428b26196d3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:41.860485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:41.875741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:41.876376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:41.876657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:41.876930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:41.887979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:41.889462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:41.904077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:41.905043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:41.912513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:41.912513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:41.912513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:41.912513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 529/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:41.912755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b6277-8c73-409f-ab37-14153273675c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:41.961381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:42.018931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b6277-8c73-409f-ab37-14153273675c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.052771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.053381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.053791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.054067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.054710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:42.055018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.056411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.073189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 536/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:42.074333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.176886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29121c-38e8-4efa-abea-c19754f8ddcc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.210375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.211006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.211313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.211638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.218972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.220429 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:42.232031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:42.232943 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:42.233875 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 530/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:42.240728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.340003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6f004-56c9-4014-9714-6dc0f38b31b1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.342372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:42.383565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.384190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.384471 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.384746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.385479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:42.385786 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.387230 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.410345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 537/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:42.411699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.444023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:42.514061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20882b4-505d-416a-8386-c0ee2b2b4332 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.547435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.548063 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.548340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.548610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.559247 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.560689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:42.574762 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:42.575754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:42.576648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:42.576648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:42.576648 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:42.579283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 531/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:42.582105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.683984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f808a-0cfd-44c0-b972-6a46803a8b9b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.718034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.718637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.718913 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.719205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.719838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:42.720140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.721513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.737006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 538/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:42.738192 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.831497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:42.850498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc623-be31-4866-a813-55e64cb42e05 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:42.885511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:42.886140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.886417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.886682 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.893498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:42.894928 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7973d37d-a7ea-41bc-be76-9dd619079f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:42.906229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:42.907132 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:42.909110 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:42.909821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 532/1070] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:42.913597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:42.932114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083bce2-8335-4531-b211-ed24eea7d689 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.932397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083bce2-8335-4531-b211-ed24eea7d689 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.932673 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083bce2-8335-4531-b211-ed24eea7d689 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.934126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 533/1071] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:42.939291 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:42.939590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:42.939913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:42.941599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:42.959701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:42.961241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:43.022954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd3eb-a4af-4218-915e-eb0006a130f2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.039195 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 534/1072] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:43.057733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.058341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.058618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.058888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.059596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:43.059904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.061328 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.078273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 539/1073] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:43.079633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.155003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:43.181739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6a33-3e54-4c0e-b70e-bba6798f05b5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.222163 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.222843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.223120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.223434 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/6ee1bf77ba614b2c88de15ad3bd3d556` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.225633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=2c5eae01c70b440e95b480d51780dd3e, role_id=6ee1bf77ba614b2c88de15ad3bd3d556)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.227076 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afed28-b0e3-4859-b42e-5e1569d4a2c7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.241969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 535/1074] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:43 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/2c5eae01c70b440e95b480d51780dd3e/roles/6ee1bf77ba614b2c88de15ad3bd3d556 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:17:43.309845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.332718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0604f91-214d-409a-9f24-b5c0e320c75d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.333086 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0604f91-214d-409a-9f24-b5c0e320c75d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.333332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0604f91-214d-409a-9f24-b5c0e320c75d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.335345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 536/1075] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:17:43.341161 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.341692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.341970 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.360827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:43.362651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:43.407932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:43.436194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538441-8483-4b66-b3ef-cbcf5ad79f32 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.444313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 537/1076] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:43.473737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.474374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.474652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.474922 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.485083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:43.489931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:43.490831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:43.495984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 540/1077] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:43.497179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.570331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:43.610368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb898d-7c6b-4b1b-b837-25989c1e4847 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.644764 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.645454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.645801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.646140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.646865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:43.647273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.648689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.661754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 538/1078] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:43.663866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.723557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.771845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ade58-2735-4bda-ae3e-23ba09d61a25 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.807868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.808513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.808794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.809075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] PATH_INFO: `/v3/users/40a15daa71214f7bba26ca3794093473` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.810316 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] RBAC: Authorizing `identity:get_user(user_id=40a15daa71214f7bba26ca3794093473)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.811893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.818041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 541/1079] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:43 2022] GET /identity/v3/users/40a15daa71214f7bba26ca3794093473 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:43.819798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:43.830096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea46d6-09d0-4e0e-b6a2-37b7e94c9473 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.833719 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:43.855432 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:43.865312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.865923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.866265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.866545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.930457 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98b861-79e8-4a9d-9ecd-6f60dc2a2809 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:43.934958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.936457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.969317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:43.969952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:43.970235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:43.970504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:43.971753 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:43.973297 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:43.974924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 542/1080] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:17:43 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:43.980976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:44.047184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 539/1081] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:44.048975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:44.071259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:44.075210 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:44.100363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94dba-d888-4b1c-85fa-84766d6659b2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:44.140053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:44.140725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:44.141007 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:44.141287 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f/users/40a15daa71214f7bba26ca3794093473/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:44.143845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] RBAC: Authorizing `identity:create_grant(project_id=e6c24f2822d141be931ddcb8c21d5b5f, user_id=40a15daa71214f7bba26ca3794093473, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:44.145277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:44.157312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01d912-b192-494f-886e-f923184d2a93 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:44.167396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 543/1082] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:43 2022] PUT /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f/users/40a15daa71214f7bba26ca3794093473/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 03:17:44.204833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:44.205957 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:44.206238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:44.206511 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:44.217917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:44.219430 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-cdfbd034-7d1d-4839-bce7-4487ab20eec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:44.235410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:44.236425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:44.237368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:44.237368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:44.237368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:44.237368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 540/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:44.242938 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:44.355868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945ec2c-7755-4933-8ed9-fe445ee7c62e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:44.396234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:44.396842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:44.397120 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:44.397386 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:44.398019 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:44.398321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:44.399701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d78a5-06be-477f-8a89-138001cd36fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:44.407124 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:44.413969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 544/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:45.401839 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3429e-e807-4def-a5ad-d616b3c2844e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:45.402336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3429e-e807-4def-a5ad-d616b3c2844e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:45.402435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3429e-e807-4def-a5ad-d616b3c2844e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:45.403735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 541/1085] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:45.408813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:45.409202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:45.409358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:45.427702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:45.428871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:45.492568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 545/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:45.755567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3429e-e807-4def-a5ad-d616b3c2844e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:45.832953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:45.854993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3429e-e807-4def-a5ad-d616b3c2844e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:45.887122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:45.887767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:45.888043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:45.888313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:45.899018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:45.903788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:45.904940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:45.905894 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 542/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 03:17:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:17:45.910907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.009272 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34de3-7fed-443f-b5e8-fb56c6a27e33 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.034667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.043046 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.043682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.043964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.044239 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.044899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:46.045206 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.046566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.054998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 546/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:17:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:46.060870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.158663 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3edec-8530-45a9-ab2e-e26c48b5afa6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.192236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.192865 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.193147 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.193421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.200427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.204912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.206390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-2201a691-5438-493c-a95f-5ff5be526778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:46.220227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:46.221167 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:46.222044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:46.222044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:46.222044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:46.222044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 543/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:46.227686 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.328801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d5e7f-1937-49eb-95d3-ad179e0f6f88 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.348457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996c01-162f-4f56-99bd-5fe12ee843c8 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.348742 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996c01-162f-4f56-99bd-5fe12ee843c8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.349017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996c01-162f-4f56-99bd-5fe12ee843c8 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.350348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 544/1090] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:46.355330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.355620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.355896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.362406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.363213 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.363503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.363771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.364412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:46.364716 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.366054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.374721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:46.375950 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:46.378710 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 547/1091] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:46.384196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.432574 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.447524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 545/1092] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:17:46.481462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0b31-d47b-4429-8b6c-1eecd3044f2e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.514309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.514933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.515249 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.515529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.522284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.523754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:17:46.535736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:46.536816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:46.537784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 548/1093] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:46.542546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.639854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38b4db-5c58-4a36-a8cf-853f7eb88bc7 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.659373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.674849 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.675512 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.675791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.676060 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.676778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:46.677347 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.678765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.695294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 546/1094] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:46.701373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.716975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.802458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf493004-b08f-4551-994f-fd38ced3534a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.812650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.815660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eb6a-bfac-4d6f-a9b0-3a2949a65ebb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:46.836325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.836924 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.837197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.837466 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.848910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:46.849532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:46.849805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:46.850079 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:46.859828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:17:46.865181 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:46.866096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:17:46.866915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 547/1095] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:46.872075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:46.904712 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:46.906090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:46.910228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:46.969771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a30f8c-4eac-4a7d-b06c-47f6c4b47206 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.002615 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.003249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.003535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.003810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.004272 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 549/1096] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:47.004545 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:47.004856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.006308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.009628 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.019599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 548/1097] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:17:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:47.020598 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.021174 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.110154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad334ec-2f8a-4180-8db3-41e3a76879e8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.115522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.118612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91e6d7-d74f-4ab3-87a9-76ae06d9682e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.144517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.145156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.145433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.145709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.152308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.152931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.153212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.153487 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] PATH_INFO: `/v3/users/40a15daa71214f7bba26ca3794093473` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.154621 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] RBAC: Authorizing `identity:get_user(user_id=40a15daa71214f7bba26ca3794093473)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.155776 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.156099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.157156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.157712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 549/1098] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/users/40a15daa71214f7bba26ca3794093473 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:47.163883 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:17:47.173465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:47.175103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:17:47.176052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:17:47.176052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:17:47.176052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:47.176052 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 550/1099] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:47.180869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.244172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.263789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c7cc-f981-4dec-9fbe-9fc8be10e719 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.284219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d0ba3-7905-43fa-9bea-fc9fd22e835e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.296651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.297258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.297535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.297812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.318000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.318605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.318882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.319170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.319811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:47.320115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.321472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.332953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.334144 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 551/1100] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:47.340018 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.365924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.367412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.440492 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb43ee-73fc-4f91-9aa2-9da60b0585e1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.462211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.465256 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 550/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:17:47.471626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.478706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.479371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.479655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.479933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.487211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.488642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:17:47.499605 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:17:47.500565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:17:47.501454 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:17:47.502145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 552/1102] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:17:47.506373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.536383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.568740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07005f1-a3ee-413a-8b2c-0cc5488ac52c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.602419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.603041 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.603343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.603626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.603865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083444b7-d044-4626-8807-fd99f20de6f8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.604737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.606177 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.607974 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 551/1103] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:47.613800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.637031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.637639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.637921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.638200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.638890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:47.639233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.640655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.654170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 553/1104] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:47.659941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:47.706840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.714783 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff87e-735f-4cfc-81c5-f9e6cdef52ee None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.748714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.749331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.749609 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.749882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.750605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f, user.id=40a15daa71214f7bba26ca3794093473) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:47.750920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.753022 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118d92d-67e1-41d7-886f-78389a1d3559 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.755859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:47.759276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 552/1105] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/role_assignments?user.id=40a15daa71214f7bba26ca3794093473&scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:47.766595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e7c8f-0833-4eb8-9493-4faeabbba2b5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:47.803112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:47.803746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:47.804026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:47.804296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:47.805028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] RBAC: Adding query filter params (role.id=6ee1bf77ba614b2c88de15ad3bd3d556, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=2c5eae01c70b440e95b480d51780dd3e) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:47.805341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:47.806765 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8022e-d900-4777-8571-2d14eaf80678 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:47.813289 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 554/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:47 2022] GET /identity/v3/role_assignments?user.id=2c5eae01c70b440e95b480d51780dd3e&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=6ee1bf77ba614b2c88de15ad3bd3d556 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:17:49.944343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e5a2-4d05-4655-8bf6-dbaee239d789 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:49.944898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e5a2-4d05-4655-8bf6-dbaee239d789 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:49.944990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e5a2-4d05-4655-8bf6-dbaee239d789 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:49.946352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 553/1107] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:49.951601 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:49.952010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:49.952151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:49.970278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:49.972103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:50.035622 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 555/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:50.300877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737e5a2-4d05-4655-8bf6-dbaee239d789 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:50.400681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737e5a2-4d05-4655-8bf6-dbaee239d789 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:50.433022 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:50.433675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:50.433957 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:50.434232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:50.435390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:50.436822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7e3da-b76b-44f5-abca-3730dc55e997 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:50.438376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 554/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:17:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:50.444626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:50.477711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:50.538992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daea5e-500f-4309-827d-850d0fd507d9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:50.572125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:50.572720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:50.573001 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:50.573275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:50.573931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:50.575365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb128bb-dabd-4a31-b802-e0522901c97f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:50.595115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 556/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:17:50 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:17:52.752352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64f8d-34a2-4f03-896d-cf9806dcbe3c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:52.752862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64f8d-34a2-4f03-896d-cf9806dcbe3c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:52.752862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64f8d-34a2-4f03-896d-cf9806dcbe3c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:52.754133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 555/1111] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:52.759411 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:52.759825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:52.759968 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:52.778136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:52.779341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:52.842967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 557/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:53.109159 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a64f8d-34a2-4f03-896d-cf9806dcbe3c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:53.174359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:53.206889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a64f8d-34a2-4f03-896d-cf9806dcbe3c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:53.239374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:53.239996 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:53.240273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:53.240544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] PATH_INFO: `/v3/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:53.241925 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] RBAC: Authorizing `identity:get_role(role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:53.243331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:53.244709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 556/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:53 2022] GET /identity/v3/roles/4363620f6a3647d0a382928ffef15ec7 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:53.250468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:53.274631 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:53.348034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7148cc-da2d-4d6c-b7d5-cdc5a0995a70 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:53.381325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:53.381960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:53.382241 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:53.382518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] PATH_INFO: `/v3/users/01637999bde745c28876edf123796efb` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:53.395936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] RBAC: Authorizing `identity:get_user(user_id=01637999bde745c28876edf123796efb)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:53.397369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:53.398945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 558/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:53 2022] GET /identity/v3/users/01637999bde745c28876edf123796efb => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:53.404660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:53.500522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe15f97-5441-4c87-844d-ab56f0f8920c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:53.533431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:53.534023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:53.534301 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:53.534573 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:53.537808 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:53.599654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:53.601080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:53.694275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 557/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:17:53.700093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:53.778250 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:53.797577 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fad84-1682-442e-aa70-75b9d047b52f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:53.830308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:53.830923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:53.831229 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:53.831504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:53.832606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:53.834008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b06665-9d40-4dad-b27b-faf96b704cda None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:53.835609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 559/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:17:53 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:53.841534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:53.874939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:53.937248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ef6de-7043-4bfb-a007-b56b6016b342 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:53.970469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:53.971003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:53.971312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:53.971588 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:53.972299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f, user.id=01637999bde745c28876edf123796efb) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:17:53.972608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:53.974023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dba70-5f58-4106-a77f-aa297a4f826d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:53.979178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 558/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:17:53 2022] GET /identity/v3/role_assignments?user.id=01637999bde745c28876edf123796efb&scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:56.062963 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad725ce-1daf-48b1-ba3f-ab15d598564d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.063541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad725ce-1daf-48b1-ba3f-ab15d598564d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.063541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad725ce-1daf-48b1-ba3f-ab15d598564d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.065419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 560/1118] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:56.070569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.070991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.071112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.088838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:56.090002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:56.153667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 559/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:56.422729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad725ce-1daf-48b1-ba3f-ab15d598564d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:56.520472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad725ce-1daf-48b1-ba3f-ab15d598564d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:56.552773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:56.553400 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.553655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.553926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] PATH_INFO: `/v3/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.555324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] RBAC: Authorizing `identity:get_role(role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:56.556667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:56.557933 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 561/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:56 2022] GET /identity/v3/roles/4363620f6a3647d0a382928ffef15ec7 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:56.563640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:56.592688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:56.660557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bb290-ae80-4564-a8d6-8132a6df78e4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:56.697603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:56.698233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.698515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.698791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] PATH_INFO: `/v3/users/01637999bde745c28876edf123796efb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.699930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=01637999bde745c28876edf123796efb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:56.707508 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:56.709088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 560/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:56 2022] GET /identity/v3/users/01637999bde745c28876edf123796efb => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:56.714801 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:56.732813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:56.814641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534fc-3c3c-4275-90e1-bacba40f3746 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:56.847091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:56.847735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.848010 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.848286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.849378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:56.850783 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac7464-9b04-4ce2-90d5-b06af35199e6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:56.852388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 562/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:17:56 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:56.858403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:56.951052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:56.955731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5114-e6e8-4e66-81fc-b00d4272c8a5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:56.988105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:56.988747 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:56.989027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:56.989303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f/users/01637999bde745c28876edf123796efb/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:56.991459 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6c24f2822d141be931ddcb8c21d5b5f, user_id=01637999bde745c28876edf123796efb, role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:56.992882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d62f7-340c-4f19-a215-409d3b55be26 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:57.005248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 561/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:17:56 2022] PUT /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f/users/01637999bde745c28876edf123796efb/roles/4363620f6a3647d0a382928ffef15ec7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:17:59.092358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037af6-2416-4a4c-8608-8fe2daefaf56 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:59.092914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037af6-2416-4a4c-8608-8fe2daefaf56 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:59.092914 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037af6-2416-4a4c-8608-8fe2daefaf56 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:59.094236 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 563/1124] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:17:59.099932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:59.100360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:59.100497 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:59.119018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:17:59.120229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:17:59.183728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 562/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:17:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:17:59.453462 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82037af6-2416-4a4c-8608-8fe2daefaf56 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:59.535160 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:59.552317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82037af6-2416-4a4c-8608-8fe2daefaf56 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:59.584532 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:59.585142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:59.585419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:59.585691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] PATH_INFO: `/v3/roles/4363620f6a3647d0a382928ffef15ec7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:59.587057 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] RBAC: Authorizing `identity:get_role(role_id=4363620f6a3647d0a382928ffef15ec7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:59.588433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:59.589699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 564/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:59 2022] GET /identity/v3/roles/4363620f6a3647d0a382928ffef15ec7 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:59.595400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:59.679660 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:59.692859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52c855-d2ce-4ba3-bde8-0de563bebcde None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:59.725630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:59.726252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:59.726534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:59.726808 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] PATH_INFO: `/v3/users/01637999bde745c28876edf123796efb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:59.727954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] RBAC: Authorizing `identity:get_user(user_id=01637999bde745c28876edf123796efb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:59.729375 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:17:59.730909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 563/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:17:59 2022] GET /identity/v3/users/01637999bde745c28876edf123796efb => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:17:59.736907 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:17:59.831040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5ec0-18ba-44dc-9b8b-e3ff00208ac6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:17:59.863038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:17:59.863650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:17:59.863927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:17:59.864197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:17:59.884425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:17:59.928478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:17:59.929937 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:00.024187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 565/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:17:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:00.030088 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:00.125536 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa72a-833b-4cc1-83ba-b04c5f19bb3c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:00.158757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:00.159412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:00.159696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:00.159971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] PATH_INFO: `/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:00.161085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] RBAC: Authorizing `identity:get_project(project_id=e6c24f2822d141be931ddcb8c21d5b5f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:00.162515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f2f3f-022d-4f10-a5e4-bb59d983db42 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:00.164134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 564/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:00 2022] GET /identity/v3/projects/e6c24f2822d141be931ddcb8c21d5b5f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:00.170187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:00.220342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:00.265382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527aa6e-6e4a-45b1-93a8-98c4196e852a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:00.297355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:00.297936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:00.298700 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:00.298969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:00.299705 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] RBAC: Adding query filter params (role.id=4363620f6a3647d0a382928ffef15ec7, scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f, user.id=01637999bde745c28876edf123796efb) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:00.300005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:00.301391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c35be-c388-4450-a616-a32ab0132fad None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:00.307068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 566/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:18:00 2022] GET /identity/v3/role_assignments?user.id=01637999bde745c28876edf123796efb&scope.project.id=e6c24f2822d141be931ddcb8c21d5b5f&role.id=4363620f6a3647d0a382928ffef15ec7 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:02.460683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68635c-34ff-43d5-944c-740f483587b5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:02.461180 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68635c-34ff-43d5-944c-740f483587b5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:02.461272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68635c-34ff-43d5-944c-740f483587b5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:02.462554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 565/1131] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:02.467867 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:02.468296 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:02.468424 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:02.486622 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:02.487964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:02.552670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 567/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:18:02.828048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68635c-34ff-43d5-944c-740f483587b5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:02.853293 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:02.928114 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68635c-34ff-43d5-944c-740f483587b5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:02.960715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:02.961330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:02.961606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:02.961874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:02.962965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:02.964414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde46844-27c1-489d-9ff4-bfc6f604915c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:02.965944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 566/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:18:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:02.973709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:03.024651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:03.075622 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ca093-da1e-4ade-9814-13e3461d757d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:03.109611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:03.110142 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:03.110421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:03.110720 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:03.111417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:03.112839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c6eff-2abd-4577-9f41-f867ea4d9e0d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:03.138870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 568/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:18:02 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:18:05.276938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062331-d676-441f-ba18-10d7e549a1da None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:05.277567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062331-d676-441f-ba18-10d7e549a1da None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:05.277567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062331-d676-441f-ba18-10d7e549a1da None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:05.278799 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 567/1135] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:05.284430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:05.284848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:05.285008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:05.303575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:05.304734 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:05.373550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 569/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:05.637831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062331-d676-441f-ba18-10d7e549a1da None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:05.726937 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:05.737557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062331-d676-441f-ba18-10d7e549a1da None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:05.770877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:05.771527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:05.771810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:05.772085 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:05.773196 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:05.774633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441a545-c6d8-4a54-bdfa-d02a36e2af96 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:05.776220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 568/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:05.781901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:05.808910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:05.878468 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7b2b0-b2ce-4d47-a4f5-83b18cf48914 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:05.911396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:05.911939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:05.912223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:05.912502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:05.913151 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:05.914571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266018d-d907-42b4-98f1-507cf7bb3cc2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:05.934835 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 570/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:18:05 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:18:08.086298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777e32-4f7d-46c8-ae8a-3efd73114a6c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:08.086827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777e32-4f7d-46c8-ae8a-3efd73114a6c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:08.086827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777e32-4f7d-46c8-ae8a-3efd73114a6c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:08.088190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 569/1139] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:08.093451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e89de-099f-418b-a029-acf08478dfce None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:08.093839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e89de-099f-418b-a029-acf08478dfce None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:08.094004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e89de-099f-418b-a029-acf08478dfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:08.112910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-804e89de-099f-418b-a029-acf08478dfce None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:08.114132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-804e89de-099f-418b-a029-acf08478dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:08.179302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 571/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:08.444886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777e32-4f7d-46c8-ae8a-3efd73114a6c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:08.542728 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777e32-4f7d-46c8-ae8a-3efd73114a6c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:08.576400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:08.577017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:08.577298 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:08.577574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:08.587454 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:08.592308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:08.593456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:08.594552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 570/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:08.599412 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e89de-099f-418b-a029-acf08478dfce None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:08.614221 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:08.695558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e89de-099f-418b-a029-acf08478dfce None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:08.727475 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:08.728347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:08.728626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:08.728901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:08.729557 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:08.729868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:08.731248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:08.739423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 572/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:08.745134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:08.767946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:08.843160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8cee4-1516-44ce-9b32-4e143aa6582d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:08.876367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:08.876971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:08.877245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:08.877520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] PATH_INFO: `/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:08.889947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] RBAC: Authorizing `identity:get_user(user_id=628bcea2d04f44bcbd31bf1f4e0448c1)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:08.891424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:08.892983 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 571/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:08 2022] GET /identity/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:08.898596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:08.974032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:08.996848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f88c6-8576-491f-9b2e-2e4daba0eb97 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:09.028538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:09.029140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:09.029422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:09.029697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:09.093941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:09.095925 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:09.167099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:09.190582 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 573/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:18:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:09.196549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:09.272492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:09.293387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77ff0-53ff-4a6c-9726-11597a85c47b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:09.326803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:09.327443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:09.327722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:09.327990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] PATH_INFO: `/v3/projects/71917a630d5744a99a9ba161dba321ef` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:09.329614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] RBAC: Authorizing `identity:get_project(project_id=71917a630d5744a99a9ba161dba321ef)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:09.331021 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0ee4b-d2ee-4c30-a381-c945db78f616 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:09.332626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 572/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:09 2022] GET /identity/v3/projects/71917a630d5744a99a9ba161dba321ef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:09.339826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:09.436759 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106338-0d32-48bd-bb42-d89ffcb87c69 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:09.469090 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:09.470270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:09.470549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:09.470820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:09.471561 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=71917a630d5744a99a9ba161dba321ef, user.id=628bcea2d04f44bcbd31bf1f4e0448c1) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:09.471871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:09.473274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c878e5-aa38-448c-9591-c2c7bea57357 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:09.478404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 574/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:18:09 2022] GET /identity/v3/role_assignments?user.id=628bcea2d04f44bcbd31bf1f4e0448c1&scope.project.id=71917a630d5744a99a9ba161dba321ef&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:11.575020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b73b-2920-41b4-9ae2-47c6b2afeb99 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:11.575678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b73b-2920-41b4-9ae2-47c6b2afeb99 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:11.575678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b73b-2920-41b4-9ae2-47c6b2afeb99 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:11.576932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 573/1147] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:11.582517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:11.582882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:11.583073 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:11.601358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:11.602549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:11.635217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:11.668332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 575/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:11.935384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70b73b-2920-41b4-9ae2-47c6b2afeb99 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:12.033030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70b73b-2920-41b4-9ae2-47c6b2afeb99 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:12.065782 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:12.066401 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:12.066685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:12.066958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:12.076661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:12.082145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:12.083321 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:12.084190 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 574/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:12.089280 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:12.182665 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2251f-9a1d-4f95-a1e6-bb503dc9f096 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:12.215562 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:12.216094 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:12.216369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:12.216637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:12.217283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:12.217584 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:12.218927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:12.226565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 576/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:12.232435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:12.242064 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:12.330148 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb15199-5e7a-490d-86a2-a2d9430e2176 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:12.362818 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:12.363457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:12.363741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:12.364014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] PATH_INFO: `/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:12.365124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] RBAC: Authorizing `identity:get_user(user_id=628bcea2d04f44bcbd31bf1f4e0448c1)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:12.366543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:12.368106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 575/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:12 2022] GET /identity/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:12.373809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:12.405370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:12.472851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc901a-a310-4ae0-bf29-b593d28f11bc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:12.505500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38546575-054b-491f-8188-e7c205115f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:12.506125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546575-054b-491f-8188-e7c205115f25 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:12.506407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546575-054b-491f-8188-e7c205115f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:12.506683 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546575-054b-491f-8188-e7c205115f25 None admin] PATH_INFO: `/v3/projects/71917a630d5744a99a9ba161dba321ef` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:12.507813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546575-054b-491f-8188-e7c205115f25 None admin] RBAC: Authorizing `identity:get_project(project_id=71917a630d5744a99a9ba161dba321ef)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:12.509235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546575-054b-491f-8188-e7c205115f25 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:12.510846 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 577/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:12 2022] GET /identity/v3/projects/71917a630d5744a99a9ba161dba321ef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:12.516912 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:12.588968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:12.613277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b9fa-f6b8-4ad7-b023-3ed6c330b474 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:12.647104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:12.647779 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:12.648061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:12.648336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] PATH_INFO: `/v3/projects/71917a630d5744a99a9ba161dba321ef/users/628bcea2d04f44bcbd31bf1f4e0448c1/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:12.650208 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] RBAC: Authorizing `identity:create_grant(project_id=71917a630d5744a99a9ba161dba321ef, user_id=628bcea2d04f44bcbd31bf1f4e0448c1, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:12.651666 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47be5b-2be1-45eb-85af-e8407e1df4bb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:12.664638 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 576/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:18:12 2022] PUT /identity/v3/projects/71917a630d5744a99a9ba161dba321ef/users/628bcea2d04f44bcbd31bf1f4e0448c1/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:18:14.769934 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453a70-3397-4f72-afe9-f08755b3072f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:14.770441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453a70-3397-4f72-afe9-f08755b3072f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:14.770441 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453a70-3397-4f72-afe9-f08755b3072f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:14.771743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 578/1154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:14.776934 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:14.777300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:14.777486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:14.795719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:14.796882 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:14.860065 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 577/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:15.129840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453a70-3397-4f72-afe9-f08755b3072f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:15.224813 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:15.228724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453a70-3397-4f72-afe9-f08755b3072f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:15.261319 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:15.261932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:15.262212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:15.262486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:15.272648 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:15.277452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:15.278533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:18:15.279460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 579/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:15.284470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:15.344008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:15.384674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0f1f3-15cd-469f-9d86-e14c271b8803 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:15.417707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:15.418308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:15.418588 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:15.418861 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:15.419552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:15.419857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:15.421225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:15.429645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 578/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:15.435689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:15.531729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32445cf0-70c8-4b12-bf6f-b4496fcc974e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:15.564355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:15.564966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:15.565247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:15.565523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] PATH_INFO: `/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:15.566626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] RBAC: Authorizing `identity:get_user(user_id=628bcea2d04f44bcbd31bf1f4e0448c1)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:15.568072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:15.569598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 580/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:15 2022] GET /identity/v3/users/628bcea2d04f44bcbd31bf1f4e0448c1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:15.575197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:15.672311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50539ef7-8673-4d8f-b01f-585b7a44ab7b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:15.698112 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:15.710853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:15.711496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:15.711781 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:15.712055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:15.777164 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:15.778608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:15.872003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 579/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:15.877948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:15.905446 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:15.975517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807d7bc-2379-4379-b901-cda2d5868795 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:16.007886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:16.008517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:16.008802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:16.009079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] PATH_INFO: `/v3/projects/71917a630d5744a99a9ba161dba321ef` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:16.010194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=71917a630d5744a99a9ba161dba321ef)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:16.011634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115403e-63d3-4ba6-a1ac-60e6f3d07ab6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:16.013209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 581/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:15 2022] GET /identity/v3/projects/71917a630d5744a99a9ba161dba321ef => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:16.019940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:16.046756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:16.116908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e54c5-caa6-4238-8321-bfdf99a47e40 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:16.149102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:16.149689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:16.149966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:16.150238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:16.150949 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=71917a630d5744a99a9ba161dba321ef, user.id=628bcea2d04f44bcbd31bf1f4e0448c1) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:16.151286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:16.152698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f997-368f-4477-a10f-2421a6cf24ac None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:16.158148 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 580/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:18:16 2022] GET /identity/v3/role_assignments?user.id=628bcea2d04f44bcbd31bf1f4e0448c1&scope.project.id=71917a630d5744a99a9ba161dba321ef&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:18.281374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa134-a641-4491-b2c2-1f146f6d20f7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:18.281901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa134-a641-4491-b2c2-1f146f6d20f7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:18.281901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa134-a641-4491-b2c2-1f146f6d20f7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:18.283378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 582/1162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:18.288494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:18.288907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:18.289045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:18.308255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:18.309424 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:18.372372 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 581/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:18.637074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fa134-a641-4491-b2c2-1f146f6d20f7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:18.729567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:18.736760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fa134-a641-4491-b2c2-1f146f6d20f7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:18.776868 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:18.777480 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:18.777760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:18.778034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:18.842932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:18.844395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:18.932060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:18.938852 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 583/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:18:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:18.945351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:18.966118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:19.040004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989207c-9dcf-446d-9b2d-891ac618caba None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:19.071877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:19.072491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:19.072768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:19.073037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:19.079435 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:19.081408 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7f630bd8-3d49-4f9d-b76b-76131b2f725f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:19.092305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:19.093462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:19.094318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 582/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:18:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:19.099376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:19.194318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987eeae-2d84-42ac-ab87-26b8f9f021d8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:19.226086 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:19.226674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:19.226952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:19.227249 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:19.227882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:19.228181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:19.229523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fbffd-47a6-4b01-af49-4091ae5a7b48 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:19.236773 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 584/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 03:18:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:21.343020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afdf1b-5318-4ef6-85e6-b6d9bc8053de None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:21.343566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afdf1b-5318-4ef6-85e6-b6d9bc8053de None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:21.343566 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afdf1b-5318-4ef6-85e6-b6d9bc8053de None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:21.344909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 583/1167] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:21.350763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d056c-de1e-441e-9cfd-2cdeea140a04 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:21.351204 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d056c-de1e-441e-9cfd-2cdeea140a04 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:21.351348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d056c-de1e-441e-9cfd-2cdeea140a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:21.369084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-de1d056c-de1e-441e-9cfd-2cdeea140a04 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:21.370261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-de1d056c-de1e-441e-9cfd-2cdeea140a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:21.425180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:21.437047 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 585/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:21.705407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afdf1b-5318-4ef6-85e6-b6d9bc8053de None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:21.784489 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:21.804263 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afdf1b-5318-4ef6-85e6-b6d9bc8053de None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:21.837436 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:21.838031 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:21.838314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:21.838593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:21.839295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:21.840669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254eac8b-aca6-4b26-8ac3-7e095ecdf8f9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:21.855620 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 584/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Tue Oct 4 03:18:21 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:18:24.003547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097404e-cd07-443e-a71c-04b10d7a2297 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:24.004068 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097404e-cd07-443e-a71c-04b10d7a2297 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:24.004159 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097404e-cd07-443e-a71c-04b10d7a2297 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:24.005422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 586/1170] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:24.010691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:24.011069 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:24.011284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:24.029591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:24.030777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:24.095133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 585/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:24.358187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097404e-cd07-443e-a71c-04b10d7a2297 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:24.456606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097404e-cd07-443e-a71c-04b10d7a2297 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:24.488881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:24.489497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:24.489777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:24.490051 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] PATH_INFO: `/v3/domains/d8bcbf9a03854f26bbb41e94179f9361` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:24.500014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8bcbf9a03854f26bbb41e94179f9361)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:24.501498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be5d4c-12ba-4d2f-b77a-381565371873 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:24.503039 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 587/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 4 03:18:24 2022] GET /identity/v3/domains/d8bcbf9a03854f26bbb41e94179f9361 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:24.508816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:24.576162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:24.607204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964783d-04ad-4d7c-8605-0cd9a97861e5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:24.639546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:24.640081 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:24.640358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:24.640626 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:24.641276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:24.643280 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4548fd-4b0c-439a-b36d-8ce6d6d512d9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:24.671191 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 586/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:18:24 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:18:26.803448 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1440b3-a458-49b0-8692-1d3099eba09e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:26.803939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1440b3-a458-49b0-8692-1d3099eba09e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:26.803939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1440b3-a458-49b0-8692-1d3099eba09e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:26.805246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 588/1174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:26.810525 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:26.810946 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:26.811083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:26.830791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:26.832016 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:26.895768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 587/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:27.164536 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1440b3-a458-49b0-8692-1d3099eba09e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:27.215040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:27.264326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1440b3-a458-49b0-8692-1d3099eba09e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:27.295809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:27.296425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:27.296708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:27.296983 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] PATH_INFO: `/v3/domains/d8bcbf9a03854f26bbb41e94179f9361` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:27.298624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8bcbf9a03854f26bbb41e94179f9361)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:27.300103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e76631-b326-4c46-84e1-f4f4d7e30740 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:27.301889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 589/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 4 03:18:27 2022] GET /identity/v3/domains/d8bcbf9a03854f26bbb41e94179f9361 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:27.307427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:27.357160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:27.406265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005556e2-753a-4be6-9142-63d439c171f9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:27.443835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:27.444370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:27.444647 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:27.444917 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:27.445561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:27.446984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc61ce-b7cd-4d39-9db0-8149a41f6b75 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:27.467448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 588/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:18:27 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:18:29.641111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef757c2-f8b9-4e40-8b5e-fa50a1c38ee9 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:29.641621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef757c2-f8b9-4e40-8b5e-fa50a1c38ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:29.641621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef757c2-f8b9-4e40-8b5e-fa50a1c38ee9 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:29.642972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 590/1178] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:29.648122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:29.648487 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:29.648684 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:29.666284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:29.667484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:29.738051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 589/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:30.003252 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef757c2-f8b9-4e40-8b5e-fa50a1c38ee9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:30.103091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef757c2-f8b9-4e40-8b5e-fa50a1c38ee9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:30.118988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:30.135792 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:30.136410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:30.136689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:30.136962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:30.147650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:30.152419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:30.153499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:18:30.154544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 591/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:18:30.161507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:30.199422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:30.256832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b7c7d-9141-467d-baae-2a4f341a1171 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:30.289659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:30.290253 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:30.290532 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:30.290803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:30.291486 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:30.291793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:30.293149 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:30.300789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 590/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:30.306979 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:30.403977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd5396-1861-4eb7-b3c4-b4dc255fb67c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:30.436416 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:30.437036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:30.437338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:30.437613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] PATH_INFO: `/v3/users/f466dd519b7e4d54a461fb8eec5e021c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:30.450686 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] RBAC: Authorizing `identity:get_user(user_id=f466dd519b7e4d54a461fb8eec5e021c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:30.452138 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:30.453702 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 592/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/users/f466dd519b7e4d54a461fb8eec5e021c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:30.459696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:30.553714 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:30.556844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904157-0fba-4771-acfc-e41cd9b56a75 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:30.589561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:30.590162 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:30.590442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:30.590717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:30.656328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:30.657802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:30.762570 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:30.768635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 591/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:30.774391 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:30.807964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:30.869863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6684df-7e46-4b4b-9a6c-8333abb88616 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:30.902745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:30.903339 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:30.903621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:30.903895 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] PATH_INFO: `/v3/projects/32e87ecd53184bfd9dffee9a05818ca6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:30.904994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] RBAC: Authorizing `identity:get_project(project_id=32e87ecd53184bfd9dffee9a05818ca6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:30.906407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c562bb-d449-406f-8f7a-fcfcf1dd0198 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:30.908059 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 593/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/projects/32e87ecd53184bfd9dffee9a05818ca6 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:30.914115 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:31.008037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c063c9-8902-47cf-9b11-a8bdadb82dc3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:31.040744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:31.041330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:31.041607 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:31.041878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:31.042585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=32e87ecd53184bfd9dffee9a05818ca6, user.id=f466dd519b7e4d54a461fb8eec5e021c) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:31.042893 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:31.044332 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e6478-b4bb-4214-a4cf-a09ce255fac2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:31.049385 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 592/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:18:30 2022] GET /identity/v3/role_assignments?user.id=f466dd519b7e4d54a461fb8eec5e021c&scope.project.id=32e87ecd53184bfd9dffee9a05818ca6&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:33.160694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aa5f0-927e-4b64-bdce-26941992b991 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:33.161226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aa5f0-927e-4b64-bdce-26941992b991 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:33.161226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aa5f0-927e-4b64-bdce-26941992b991 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:33.162543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 594/1186] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:33.167657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:33.168086 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:33.168202 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:33.186695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:33.187892 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:33.243572 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:33.252643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 593/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:33.526342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147aa5f0-927e-4b64-bdce-26941992b991 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:33.624386 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147aa5f0-927e-4b64-bdce-26941992b991 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:33.626652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:33.657817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:33.658428 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:33.658706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:33.658979 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:33.668681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:33.674006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:33.675104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:18:33.676592 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 595/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:33.681215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:33.776671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f308c-0d09-4f52-aa8d-ddc7482847cb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:33.810772 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:33.811406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:33.811691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:33.811963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:33.812619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:33.812926 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:33.814831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:33.822602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 594/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:33.828273 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:33.924371 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06256e-5815-47e4-93bb-23f6a80bb4cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:33.957602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:33.958207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:33.958485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:33.958755 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] PATH_INFO: `/v3/users/f466dd519b7e4d54a461fb8eec5e021c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:33.959876 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=f466dd519b7e4d54a461fb8eec5e021c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:33.961282 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:33.962793 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 596/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:33 2022] GET /identity/v3/users/f466dd519b7e4d54a461fb8eec5e021c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:33.968387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:34.022795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:34.066373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a7f-6685-45ac-b77b-cbcda7eddd2e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:34.098103 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:34.098725 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:34.099006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:34.099305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] PATH_INFO: `/v3/projects/32e87ecd53184bfd9dffee9a05818ca6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:34.100416 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] RBAC: Authorizing `identity:get_project(project_id=32e87ecd53184bfd9dffee9a05818ca6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:34.101838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a921fe-d123-4eda-a4af-a0d1d015413c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:34.103457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 595/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:33 2022] GET /identity/v3/projects/32e87ecd53184bfd9dffee9a05818ca6 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:34.109347 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:34.140366 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:34.207509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fb9e0-b5e0-453b-a0ed-6f970caa3dd3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:34.240173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:34.240828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:34.241112 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:34.241389 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] PATH_INFO: `/v3/projects/32e87ecd53184bfd9dffee9a05818ca6/users/f466dd519b7e4d54a461fb8eec5e021c/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:34.243292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] RBAC: Authorizing `identity:create_grant(project_id=32e87ecd53184bfd9dffee9a05818ca6, user_id=f466dd519b7e4d54a461fb8eec5e021c, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:34.244715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa1adc-27d1-45e4-bc54-506f9cab41ce None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:34.258223 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 597/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:18:34 2022] PUT /identity/v3/projects/32e87ecd53184bfd9dffee9a05818ca6/users/f466dd519b7e4d54a461fb8eec5e021c/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:18:36.351310 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d533-5e2a-47b5-88e6-881c1de6e5b1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:36.351947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d533-5e2a-47b5-88e6-881c1de6e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:36.351947 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d533-5e2a-47b5-88e6-881c1de6e5b1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:36.353173 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 596/1193] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:18:36.358304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:36.358694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:36.358859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:36.377899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:18:36.379088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:18:36.443802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 598/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:18:36.714602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192d533-5e2a-47b5-88e6-881c1de6e5b1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:36.812194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192d533-5e2a-47b5-88e6-881c1de6e5b1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:36.835134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:36.845804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:36.846418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:36.846697 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:36.846968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:36.856756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:18:36.861590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:18:36.862778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:18:36.863899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 597/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:18:36.868542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:36.892602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:36.966378 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cce8fb-fb7c-4a6a-8bda-4ce1ce108cab None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:36.999076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:36.999705 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:36.999990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:37.000267 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:37.001511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:37.001819 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:37.003212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:37.010921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 599/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:37.016640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:37.115248 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2a5f-8972-4e12-bd60-2e1e670b8083 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:37.148058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:37.148678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:37.148957 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:37.149228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] PATH_INFO: `/v3/users/f466dd519b7e4d54a461fb8eec5e021c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:37.150336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] RBAC: Authorizing `identity:get_user(user_id=f466dd519b7e4d54a461fb8eec5e021c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:37.151787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:37.153325 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 598/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 03:18:37 2022] GET /identity/v3/users/f466dd519b7e4d54a461fb8eec5e021c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:37.159578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:37.239599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:37.255368 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905427c-a684-460e-be8e-5fc946b4c315 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:37.286997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:37.287618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:37.287899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:37.288171 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:37.351809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:37.353225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:37.433205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:37.449872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 600/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:18:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:18:37.456348 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:37.507809 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:37.554066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5613e4-3084-4983-a589-1b77f3de62bf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:37.587052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:37.587692 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:37.587969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:37.588238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] PATH_INFO: `/v3/projects/32e87ecd53184bfd9dffee9a05818ca6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:37.589333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] RBAC: Authorizing `identity:get_project(project_id=32e87ecd53184bfd9dffee9a05818ca6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:37.590757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c38941-e29a-4af6-9b1f-6c0262a07b5b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:37.592386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 599/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 03:18:37 2022] GET /identity/v3/projects/32e87ecd53184bfd9dffee9a05818ca6 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:37.598549 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:18:37.694522 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f87-2e02-4403-a358-96c2582f7566 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:18:37.726261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:18:37.726848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:18:37.727125 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:18:37.727427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:18:37.728137 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=32e87ecd53184bfd9dffee9a05818ca6, user.id=f466dd519b7e4d54a461fb8eec5e021c) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:18:37.728444 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:18:37.729847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f6671-12dc-41f1-a71f-a2adf95ecb4d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:18:37.735945 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 601/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:18:37 2022] GET /identity/v3/role_assignments?user.id=f466dd519b7e4d54a461fb8eec5e021c&scope.project.id=32e87ecd53184bfd9dffee9a05818ca6&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:18:47.583017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=40 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:18:47.732996 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=76 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:33.383275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4c02-b592-47bc-893c-05a82854f5ea None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:33.383800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4c02-b592-47bc-893c-05a82854f5ea None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:33.383800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4c02-b592-47bc-893c-05a82854f5ea None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:33.385418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 600/1201] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:19:33.391406 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:33.391880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:33.391976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:33.412385 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:33.413769 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:33.505861 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 602/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:19:33.783953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea4c02-b592-47bc-893c-05a82854f5ea None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:33.944915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea4c02-b592-47bc-893c-05a82854f5ea None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:33.994193 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:33.994836 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:33.995122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:33.995426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:34.008819 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:34.010612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:34.032306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:34.033646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:34.035061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:34.035061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:34.035061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:34.036713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 601/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:34.045191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:34.184777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7b3ec-15b5-4c5f-b23f-304e43135279 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:34.216607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:34.223953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:34.224800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:34.225105 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:34.225383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:34.226111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:34.226422 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:34.227954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc052-90af-4a18-b853-3e8c1e52bdcb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:34.241729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 603/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:34.247804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:34.323422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:34.404700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a651d-bf05-4b6c-b7c8-2cab14a61fcf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:34.455669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:34.456317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:34.456629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:34.456968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:34.457708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:34.459406 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871415bb-f22f-421b-8d33-4c038fee2785 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:34.489595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 602/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 03:19:34 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:19:37.239056 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa86f9-248f-483c-846d-38465cc9bd3e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:37.239650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa86f9-248f-483c-846d-38465cc9bd3e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:37.239751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa86f9-248f-483c-846d-38465cc9bd3e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:37.241322 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 604/1206] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:19:37.247854 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:37.252277 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:37.252625 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:37.272604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:37.273795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:37.350953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 603/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:19:37.655880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa86f9-248f-483c-846d-38465cc9bd3e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:37.784493 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa86f9-248f-483c-846d-38465cc9bd3e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:37.827766 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:37.828794 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:37.829315 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:37.829777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:37.841435 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:19:37.847278 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:37.848433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:37.849346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:37.851540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 605/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:19:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:37.858526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:37.909428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:37.990580 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446f98d-9ddf-4ac0-b1d2-63d7d1e70de4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.034888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.035715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.035908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.036223 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:38.037058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:38.037404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:38.038910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:38.049749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 604/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:19:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:38.056219 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:38.119666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:38.179377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b756-7d96-4f98-864c-6a727cd235b4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.215537 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.216193 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.216472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.216746 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:38.227072 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:38.228529 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:38.243657 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:38.245227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:38.246126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:38.246126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:38.246126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:38.246126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 606/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:19:38.253330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:38.359421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:38.363053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008e39b-2708-4c4a-8794-71c8bd0bcebf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.421863 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.422477 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.422755 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.423028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:38.423713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:38.424020 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:38.425652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:38.442067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 605/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:38.448034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:38.555508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff216d1-c45a-4106-bf17-c71b35edf27a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.557485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:38.594931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.595715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.595997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.596270 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:38.606449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:38.608141 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:19:38.619540 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:38.620742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:38.621905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:38.624225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 607/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:38.629795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:38.739744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e23e00-dfb3-4240-a947-9039a6867e1f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.785595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.786235 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.786546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.786951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:38.787812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:38.788271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:38.789938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:38.810787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 606/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:19:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:38.816284 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:38.925948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b7a-2387-4474-be0b-79c24ee0cbf8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:38.969077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:38.969744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:38.969975 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:38.970298 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:39.006294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:39.049816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:39.051279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:39.168952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 608/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:19:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:19:39.177537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:39.198374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:39.289458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfb76d-f0bf-4913-a69d-7b832d9c9afa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:39.328722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:39.329484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:39.329808 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:39.330134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:39.341591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:39.343061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:39.360314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:39.361404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:39.362308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:39.362308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:39.362308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:39.362308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 607/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:39.367857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:39.435329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:39.476375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44db7-7403-41a9-b149-241d698930f9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:39.516245 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:39.516871 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:39.517170 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:39.517524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:39.518199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:39.518508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:39.520078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:39.532896 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 609/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:39.538760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:39.619791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:39.657313 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bedaaf-b01f-4aae-b138-fece37be72aa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:39.695984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:39.696621 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:39.696905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:39.697201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:39.705057 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:39.706652 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:19:39.720585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:39.721599 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:39.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 608/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:19:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:39.728046 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:39.837952 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c2bc-6f5f-4ad0-81f4-c113b9bfea38 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:39.863560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:39.879465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:39.880123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:39.880381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:39.880725 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:39.881544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:39.881915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:39.883588 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356e09-4121-4d57-b22f-dc24f4462283 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:39.897635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 610/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:19:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:39.904986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:40.020338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bd49a-c340-4995-badb-40e56e30ed27 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:40.052613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:40.059102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:40.059851 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:40.060166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:40.060462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:40.061229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=b50de3f398ee4cd487229cecf88a5b35) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:40.061578 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:40.063305 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ed2e-8ae9-4795-abd7-dd4e4a04c249 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:40.069790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 609/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:19:39 2022] GET /identity/v3/role_assignments?user.id=b50de3f398ee4cd487229cecf88a5b35&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:42.313271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0049-ac13-4133-9356-63b3901a1516 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:42.313853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0049-ac13-4133-9356-63b3901a1516 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:42.313853 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0049-ac13-4133-9356-63b3901a1516 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:42.315195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 611/1220] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:19:42.320857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:42.321368 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:42.321462 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:42.341836 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:42.343089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:42.423034 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 610/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 03:19:42.706476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0049-ac13-4133-9356-63b3901a1516 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:42.821158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0049-ac13-4133-9356-63b3901a1516 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:42.855751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:42.856439 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:42.856711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:42.856981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:42.866919 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:19:42.871747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:42.873449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:42.874548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 612/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:19:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:42.879370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:42.983812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8bd41-66e8-4b9c-bfbb-0ccfae6b588a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.007953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:43.019888 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.020521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.020788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.021061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.021723 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:43.022030 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.023431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.031945 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 611/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:19:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:43.037865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:43.089209 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:43.176177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b073225-b535-4322-b4b9-1e507b376d41 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.214998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.216302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.216636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.216948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.228518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.230102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:43.247949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:43.249102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:43.249995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:43.249995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:43.249995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:43.249995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 613/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:43.255105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:43.369494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2f2f-3bb1-468a-b788-c25ef19ae79b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.407217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.407833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.408145 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.408483 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.409267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:43.409639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.411095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.423924 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 612/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:19:43.431744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:43.538440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:43.546127 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad34388-1d77-460e-a07d-249826f572ff None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.579808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.580440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.580736 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.580997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.588613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.590053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:19:43.602021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:43.608642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:43.609752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 614/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:43.611572 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:43.661982 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:43.720028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6bb23-b72e-468e-8d7e-66341b1adc20 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.759699 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.760309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.760604 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.760879 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.761611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:43.761900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.763897 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.781798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 613/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:43.787610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:43.897447 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7513-3060-42bd-af46-27661d65b6e1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:43.933498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:43.934129 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:43.934472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:43.934752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:43.948982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:43.951538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:43.966837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:43.967929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:43.968855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:43.968855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:43.968855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:43.968855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 615/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:43.975521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:44.077226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c296e-76c9-444f-8359-e6570192a8ec None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:44.079540 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:44.113678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6de5-b635-4622-8490-963311910947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:44.114315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6de5-b635-4622-8490-963311910947 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:44.114662 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6de5-b635-4622-8490-963311910947 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:44.115040 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6de5-b635-4622-8490-963311910947 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:44.115702 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa6de5-b635-4622-8490-963311910947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:44.116053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa6de5-b635-4622-8490-963311910947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:44.117557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa6de5-b635-4622-8490-963311910947 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:44.130731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 614/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:44.136802 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:44.147250 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:44.240343 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dbdbc-c082-48c4-bc60-2ace9e00c5b9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:44.275258 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:44.275902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:44.276173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:44.276451 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:44.283781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:44.285186 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:19:44.297854 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:44.299593 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:44.300815 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 616/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:19:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:44.305045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6de5-b635-4622-8490-963311910947 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:44.422160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6de5-b635-4622-8490-963311910947 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:44.458610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:44.459250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:44.459535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:44.459869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:44.460485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:44.460816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:44.462228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0601cb-85f2-40ea-ad5a-8bbf397c2a58 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:44.474205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 615/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:19:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:44.480196 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:44.548434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:44.585619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a028-42cb-407d-9204-f6afeae6b04d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:44.620417 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:44.621074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:44.621356 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:44.621632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/b50de3f398ee4cd487229cecf88a5b35/roles/78b056129b1048b1ae7f4e40234dcd95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:44.642083 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=b50de3f398ee4cd487229cecf88a5b35, role_id=78b056129b1048b1ae7f4e40234dcd95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:44.642315 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04702d2e-edc0-44b3-8a0a-3f835ec89afd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:44.656060 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 617/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:19:44 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/b50de3f398ee4cd487229cecf88a5b35/roles/78b056129b1048b1ae7f4e40234dcd95 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:19:47.008746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676da5c-e283-4a8d-9e09-306633e58cb8 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:47.009314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676da5c-e283-4a8d-9e09-306633e58cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:47.009314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676da5c-e283-4a8d-9e09-306633e58cb8 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:47.010787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 616/1233] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:19:47.017075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:47.017548 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:47.017697 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:47.040240 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:47.041539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:47.127568 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 618/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:19:47.442986 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676da5c-e283-4a8d-9e09-306633e58cb8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:47.473910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:47.557195 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676da5c-e283-4a8d-9e09-306633e58cb8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:47.592765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:47.593397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:47.593681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:47.593953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:47.604318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:19:47.609717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:47.612523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:47.613363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 617/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 03:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:19:47.616733 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:47.664119 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:47.729202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1d30-bb0b-497f-9525-f7593a9cffb3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:47.772859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:47.773498 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:47.773785 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:47.774064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:47.774731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:47.775040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:47.776443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:47.786410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 619/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 03:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:47.794619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:47.889657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:47.909645 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07e75-5926-46da-9a1f-e42fd411a13a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:47.948522 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:47.949274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:47.949602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:47.949902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:47.962265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:47.963920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:47.981267 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:47.982311 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:47.983254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:47.983254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:47.983254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:47.983254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 618/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:47.988578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:48.100951 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ee43-1829-4d14-96ea-40b9570f4039 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:48.112082 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:48.144800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:48.145633 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:48.146079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:48.146372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:48.147235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:48.147645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:48.149638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:48.164986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 620/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:48.170944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:48.289254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc87a-5b66-42d4-9475-43b3c7af182f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:48.326629 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:48.327288 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:48.327565 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:48.327831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:48.330538 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:48.335835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:48.337422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:19:48.348644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:48.349993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:48.351415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:48.354796 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 619/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:48.356535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:48.466949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3417f-df43-4ccf-b40a-50d3d696e0bf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:48.510973 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:48.511667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:48.511953 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:48.512233 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:48.513043 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:48.513419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:48.514979 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:48.539637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 621/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:19:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:48.546507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:48.663835 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dc9b1-acad-4f65-abba-8d4c31a12b5c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:48.700899 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:48.702023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:48.702023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:48.702023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:48.768361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:48.775830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:48.777422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:48.889567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 620/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:19:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 03:19:48.896067 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:48.931923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:49.021985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53669e3f-e0aa-451a-935a-cf68aa7e9773 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:49.062791 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:49.063541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:49.063955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:49.064269 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:49.076530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:49.077988 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:49.094460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:49.103770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:49.104817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:49.104817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:49.104817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:49.104817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 622/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:49.105053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:49.219108 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:49.226035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa70807-6e5f-4f02-85d1-c7063247dfdb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:49.264720 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:49.265506 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:49.265915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:49.266294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:49.267106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:49.267549 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:49.269130 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:49.282517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 621/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:49.288479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:49.379285 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:49.403799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88414f8-441e-4d2c-8d8a-d5b0e404af7d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:49.440169 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:49.440840 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:49.441117 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:49.441387 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:49.448397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:49.449900 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:19:49.462100 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:49.463306 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:49.464691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 623/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:19:49.469828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:49.579727 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733296-e424-4825-ab9c-30934d8c8886 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:49.615919 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:49.616558 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:49.616877 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:49.617151 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:49.617911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:49.618258 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:49.619748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e09cc5-0fe8-4dfd-8c40-4f1bbce767da None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:49.634157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 622/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:19:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:49.641032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:49.755621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c51fd-6f8d-489e-a0c5-017dd41bd5c1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:49.790246 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:49.790850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:49.791126 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:49.791439 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:49.792212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] RBAC: Adding query filter params (role.id=78b056129b1048b1ae7f4e40234dcd95, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=b50de3f398ee4cd487229cecf88a5b35) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:49.792516 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:49.794541 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186575-559c-42ca-9d74-39a62c331793 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:49.806857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 624/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:19:49 2022] GET /identity/v3/role_assignments?user.id=b50de3f398ee4cd487229cecf88a5b35&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=78b056129b1048b1ae7f4e40234dcd95 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 03:19:52.349309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d6c9-a2f3-4a0c-9007-ab2da1b59caa None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:52.349890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d6c9-a2f3-4a0c-9007-ab2da1b59caa None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:52.349890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d6c9-a2f3-4a0c-9007-ab2da1b59caa None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:52.351218 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 623/1247] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:19:52.357111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:52.357556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:52.357670 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:52.365234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:52.380865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:52.381674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:52.473928 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 625/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:19:52.792698 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d6c9-a2f3-4a0c-9007-ab2da1b59caa None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:52.815417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:52.912482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d6c9-a2f3-4a0c-9007-ab2da1b59caa None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:52.951911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:52.952567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:52.952876 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:52.953193 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:52.964584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:19:52.970078 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:52.971252 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:52.972113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 624/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:19:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:52.978113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.080212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088d9d-e569-451e-818e-b0b342c5f73c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.114569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:53.121455 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.122431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.122616 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.123029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.123999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:53.124501 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.126696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:53.139091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 626/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:53.145134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.227228 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:53.251797 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6b57-c6b9-4137-aa9e-9228b678c285 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.286431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.287061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.287367 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.287641 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.298245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.299961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:53.314561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:53.315616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:53.316509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:53.316509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:53.316509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:53.316509 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 625/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:53.321514 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.425146 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e63e6b-3519-44af-98de-73729171c151 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.460589 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.461194 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.461470 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.461739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.462377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:53.462682 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.464062 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:53.476488 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 627/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:53.482754 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.581182 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022842a-ccee-4dca-8f11-9daffcc94033 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.606721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:53.614425 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.615044 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.615347 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.615619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.623380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.624805 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:19:53.635595 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:53.636683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:53.637878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 626/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:53.642717 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.696064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:53.739761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6cc90-c2b7-4f00-a916-2beb6818d8d0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.772149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.772743 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.773023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.773293 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.774004 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:53.774308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.775748 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:53.791901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 628/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:19:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:53.798008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:53.893935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4adca-1f53-4cb9-be7e-ffb15caa8949 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:53.926749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:53.927366 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:53.927644 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:53.927911 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:53.979358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:53.992077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:53.993510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.087333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 627/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:19:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:19:54.094517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:54.197811 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e9f45-c15f-4de6-989f-2933ad1aea5b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:54.211620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:54.232973 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:54.233621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:54.233899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:54.234171 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:54.244676 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:54.246113 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:54.260263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:54.261370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:54.262833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:54.262833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:54.262833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:54.262833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 629/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:19:54.267558 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:54.350684 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:54.362444 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eadda9-01a5-48c2-b877-007452348f96 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:54.395061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:54.395689 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:54.395965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:54.396234 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:54.396858 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:54.397156 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:54.398513 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.409915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 628/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:54.415749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:54.511327 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70fd0-c685-4ea8-977f-9daaa9364bfb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:54.544450 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:54.545064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:54.545340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:54.545611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:54.552227 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:54.553639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:19:54.565115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:54.566075 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:19:54.566986 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 630/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:54.572102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:54.673470 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a4bc-ac69-4142-af77-e031e56d63d5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:54.706837 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:54.707526 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:54.707829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:54.708106 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:54.708789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:54.709097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:54.710518 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9409b-93e5-4d8c-8d30-f98bd740793b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.723224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 629/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:19:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:54.729006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:54.754206 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:54.826028 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13875b-e76e-4579-9971-5981db2ae1c1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:54.858414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:54.858999 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:54.859303 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:54.859579 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:54.860290 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=b50de3f398ee4cd487229cecf88a5b35) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:54.860598 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:54.861997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f43c5c-fd66-40ca-ae8f-7d49416d180d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:54.867103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 631/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:19:54 2022] GET /identity/v3/role_assignments?user.id=b50de3f398ee4cd487229cecf88a5b35&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:56.975531 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675bfda-02cd-428f-b7a6-c697ad585adb None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:56.976049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675bfda-02cd-428f-b7a6-c697ad585adb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:56.976049 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675bfda-02cd-428f-b7a6-c697ad585adb None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:56.977390 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 630/1261] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:19:56.982982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:56.983440 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:56.983565 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:57.001340 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:19:57.002520 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:19:57.066667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 632/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:19:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:19:57.335011 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675bfda-02cd-428f-b7a6-c697ad585adb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:57.346380 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:57.436073 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675bfda-02cd-428f-b7a6-c697ad585adb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:57.469967 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:57.470582 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:57.470857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:57.471123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:57.480792 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:19:57.485552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:57.486665 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:57.487752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 631/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:19:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:57.492624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:57.510346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:57.588824 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357275e-8809-4bc3-ae56-e97ebbc951b6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:57.622699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:57.623320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:57.623599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:57.623870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:57.624515 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:57.624817 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:57.626177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:57.633760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 633/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:19:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:57.639556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:57.704610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:57.737186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71405943-30b1-4dd3-aa25-049d5b1617a6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:57.770178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:57.770785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:57.771061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:57.771356 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:57.781327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:57.782741 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:57.797244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:57.798336 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:57.799914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:57.799914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:57.799914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:57.799914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 632/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:57.804265 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:57.882530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:57.901629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb573ab-797a-4f08-ad54-388dad112bc2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:57.934939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:57.935558 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:57.935837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:57.936108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:57.936738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:57.937038 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:57.938384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:57.950029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 634/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:57.955651 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.050289 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597b31-aa49-4f4c-99ec-b0beb960e6ef None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.069352 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:58.083269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.083881 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.084158 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.084427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.092136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.093557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:19:58.106048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:58.107090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:58.108098 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 633/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:58.113487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.210132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa26915-c904-4b4d-93cb-8a900c43c24d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.242590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.243200 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.243478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.243744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.244452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:58.244752 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.246189 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.254402 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:58.263078 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 635/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:19:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:58.268761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.365466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204dac-82e5-49df-9c1e-42ae94573917 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.397467 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.398083 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.398360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.398630 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.409187 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.410628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:19:58.425095 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:58.426249 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:19:58.427345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:19:58.427345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:19:58.427345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:58.427345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 634/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:58.431780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.525758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031ecd-9560-4dda-9e8f-9ca29a145e43 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.558639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.559250 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.559546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.559799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.560423 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:58.560724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.562064 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.572721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 636/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:58.578194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.609244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:58.675529 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324113d-2788-446a-98aa-e7291cb0caca None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.707396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.708005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.708287 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.708562 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.715499 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.716938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:19:58.727896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:19:58.728905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:19:58.730017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 635/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:19:58.734727 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.790798 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:58.830809 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1045-b14f-4731-ab01-6560c6b82778 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:58.862575 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:58.863184 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:58.863461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:58.863732 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:58.864405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:19:58.864707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:58.866102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956a870-f23e-42ad-aad0-5bbbc3f6b5a6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:58.877377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 637/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:19:58.883226 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:19:58.974376 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:19:58.983073 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f06b1-c7ae-4fcf-ba94-ef9477a6d133 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:19:59.015121 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:19:59.015780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:19:59.016058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:19:59.016329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] PATH_INFO: `/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/b50de3f398ee4cd487229cecf88a5b35/roles/b5478fbd12434b87b67e120e15ebc8f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:19:59.018244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] RBAC: Authorizing `identity:create_grant(project_id=719ab2e182354971bdbb56ff51aa6ac3, user_id=b50de3f398ee4cd487229cecf88a5b35, role_id=b5478fbd12434b87b67e120e15ebc8f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:19:59.019701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3aef9-9e49-49c3-a9cb-f416ac4b563e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:19:59.032315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 636/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 03:19:58 2022] PUT /identity/v3/projects/719ab2e182354971bdbb56ff51aa6ac3/users/b50de3f398ee4cd487229cecf88a5b35/roles/b5478fbd12434b87b67e120e15ebc8f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:20:01.120266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3cd-a483-431b-9b5e-684e120716ed None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:01.121533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3cd-a483-431b-9b5e-684e120716ed None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:01.121533 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3cd-a483-431b-9b5e-684e120716ed None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:01.122747 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 638/1274] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:20:01.128528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:01.128953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:01.129084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:01.150262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:20:01.151770 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:20:01.219933 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 637/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:20:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:20:01.486472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c3cd-a483-431b-9b5e-684e120716ed None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:01.582851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c3cd-a483-431b-9b5e-684e120716ed None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:01.599508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:01.616220 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:01.616833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:01.617108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:01.617375 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:01.626982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 03:20:01.631796 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:01.632855 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:01.633692 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 639/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 03:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:01.638746 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:01.718315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:01.736900 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9360a2-50e5-4b3f-b9dc-e23b55c0e73a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:01.769678 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:01.770275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:01.770557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:01.770830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:01.771758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:01.772318 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:01.773831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:01.782232 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 638/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 03:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:01.787940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:01.884523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0285ac-021a-4970-aa4d-2961f025e401 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:01.916998 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:01.917624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:01.917903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:01.918175 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:01.928483 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:01.929901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:20:01.943473 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:01.944722 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:20:01.945936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:20:01.945936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:20:01.945936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:01.945936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 640/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:01.950534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.045530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500efe69-7101-4824-a742-f411d3dcb580 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:02.078129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:02.078716 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:02.078993 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:02.079292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:02.079915 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:02.080215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:02.081554 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:02.093002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 639/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:02.098790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.129927 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:02.200029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eae02-ea43-4d88-88c6-138603db1c6e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:02.231323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:02.231939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:02.232218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:02.232486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:02.239595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:02.241015 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 03:20:02.252359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:02.253778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:02.255238 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 641/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:20:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 03:20:02.259220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.263695 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:02.357489 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae11e-2de8-45d0-8acb-03203cda561f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:02.389639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:02.390237 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:02.390516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:02.390787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:02.391537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:02.391841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:02.393268 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:02.410007 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 640/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 03:20:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:02.415761 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.503173 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:02.513314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c226-77e8-4a5b-a5eb-61d24db378d9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:02.546103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:02.546706 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:02.546990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:02.547304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:02.613499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:02.614944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:02.699569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:02.709711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 642/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 03:20:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 03:20:02.716309 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.775452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:02.813269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edadfc-84dc-4d5a-9a9f-1c84f2b4bd9c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:02.845817 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:02.846428 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:02.846708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:02.846979 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:02.856637 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:02.858077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 03:20:02.872606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:02.873707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 03:20:02.874773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 03:20:02.874773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 03:20:02.874773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:20:02.874773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 641/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:02.879365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:02.975381 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02097b8-1de0-4f05-84ff-755313ec5b95 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:03.008401 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:03.008997 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:03.009271 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:03.009539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:03.010166 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:03.010467 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:03.012443 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:03.022985 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 643/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 03:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:03.028717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:03.126107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb4696-4a5f-4426-8b81-b0c3a023279d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:03.139634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:03.159337 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:03.159989 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:03.160271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:03.160548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:03.166984 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:03.168961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 03:20:03.180308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:03.181354 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 04 03:20:03.182639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 642/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 03:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:03.187721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:03.235414 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:03.282925 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10452c1-9ace-44a1-8a7f-9daf0b17cda3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:03.315506 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:03.316099 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:03.316376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:03.316649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:03.317325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:03.317630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:03.319033 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5764a6-3eb7-46c3-b374-cce29f4d4d36 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:03.330372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 644/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 03:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:03.336292 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:03.432890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ecc5f-2823-4b7e-9d90-4e3044d9c444 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:03.465902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:03.466496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:03.466771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:03.467041 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:03.467778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] RBAC: Adding query filter params (role.id=b5478fbd12434b87b67e120e15ebc8f6, scope.project.id=719ab2e182354971bdbb56ff51aa6ac3, user.id=b50de3f398ee4cd487229cecf88a5b35) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:03.468087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:03.469492 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9371ef9-e97e-453f-8ffe-b5243242f78e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:03.475005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 643/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 03:20:03 2022] GET /identity/v3/role_assignments?user.id=b50de3f398ee4cd487229cecf88a5b35&scope.project.id=719ab2e182354971bdbb56ff51aa6ac3&role.id=b5478fbd12434b87b67e120e15ebc8f6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:05.615300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b4a-ba95-4629-b493-a78d91f3d18f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:05.615845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b4a-ba95-4629-b493-a78d91f3d18f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:05.615845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b4a-ba95-4629-b493-a78d91f3d18f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:05.617180 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 645/1288] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:20:05.622493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:05.622890 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:05.623075 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:05.641777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:20:05.642880 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:20:05.671394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:05.708230 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 644/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:20:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:20:05.972456 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934b4a-ba95-4629-b493-a78d91f3d18f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:06.071187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934b4a-ba95-4629-b493-a78d91f3d18f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:06.081429 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:06.107190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:06.107795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:06.108158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:06.108363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:06.108931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:06.110292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:06.119388 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:06.120525 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:06.121396 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 646/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:20:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:06.127241 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:06.228493 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03407c5-cd92-4b40-b243-b7dee28073fb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:06.260587 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:06.261726 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:06.262009 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:06.262282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:06.262905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:06.263246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:06.264623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15da3-f6b7-4c25-8641-3ef84e7b2943 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:06.269278 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 645/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 03:20:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:06.275040 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:06.370363 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dad61-0f8f-4b9d-bdf1-1835ca2bfbac None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:06.404612 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:06.405210 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:06.405490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:06.405764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:06.406383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:06.406686 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:06.408077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aea66-d71c-4363-a107-f3e359c13617 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:06.412325 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 647/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 03:20:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:08.535981 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b2ed-cb15-4d74-ba38-c387df6cb51c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:08.536594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b2ed-cb15-4d74-ba38-c387df6cb51c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:08.536594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b2ed-cb15-4d74-ba38-c387df6cb51c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:08.537844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 646/1293] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:20:08.543428 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578208-57b1-4d10-bc72-f82cc1d5b722 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:08.543870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578208-57b1-4d10-bc72-f82cc1d5b722 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:08.543967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578208-57b1-4d10-bc72-f82cc1d5b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:08.562502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a6578208-57b1-4d10-bc72-f82cc1d5b722 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:20:08.563714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a6578208-57b1-4d10-bc72-f82cc1d5b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:20:08.591751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:08.628383 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 648/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:20:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:20:08.896242 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b2ed-cb15-4d74-ba38-c387df6cb51c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:08.927283 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:08.995834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b2ed-cb15-4d74-ba38-c387df6cb51c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:09.027895 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:09.028482 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:09.028766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:09.029036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:09.029685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:09.031038 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67005d43-ac26-4c6b-8552-234bf6a4893e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:09.046117 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 647/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 03:20:08 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:20:11.181235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a8d2a-ab54-4e1c-b84e-40c776052dd2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:11.181806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a8d2a-ab54-4e1c-b84e-40c776052dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:11.181806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a8d2a-ab54-4e1c-b84e-40c776052dd2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:11.184115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 649/1296] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:20:11.189935 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:11.190422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:11.190516 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:11.210643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:20:11.211931 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:20:11.285157 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 648/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:20:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:20:11.554283 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a8d2a-ab54-4e1c-b84e-40c776052dd2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:11.654652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a8d2a-ab54-4e1c-b84e-40c776052dd2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:11.687699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:11.688308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:11.688586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:11.688856 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:11.689430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:11.690779 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:11.699370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:11.700595 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:11.701508 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 650/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:20:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:11.706339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:11.724128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:11.801285 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d299-00f5-4a3c-9c12-3e10ca5c51bf None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:11.833323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:11.833916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:11.834195 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:11.834466 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:11.835087 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:11.835418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:11.836776 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:11.842143 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 649/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 03:20:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:11.847995 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:11.879814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:11.946256 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479ccd-0639-4746-873e-836bd4a81b58 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:11.978358 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:11.978949 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:11.979327 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:11.980193 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:11.980888 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] RBAC: Adding query filter params (interface=public, service_id=88e5bc4c96ee4e748f8c89280123264d, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:11.981199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:11.982547 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9555bb9-1a89-4bd7-8062-6b1c81d28cdc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:11.987398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 651/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 03:20:11 2022] GET /identity/v3/endpoints?service_id=88e5bc4c96ee4e748f8c89280123264d&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:11.992870 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:12.070361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:12.088028 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99d45c-8868-4476-b875-e1fa9458e072 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:12.122317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:12.122905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:12.123213 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:12.123484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:12.124071 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:12.125419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea9bf4-ae6e-4a32-b32c-d03509bf3bbd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:12.131921 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 650/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 03:20:11 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:20:14.234077 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd89eb-93c5-4f70-976e-5e31602e7669 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:14.234632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd89eb-93c5-4f70-976e-5e31602e7669 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:14.234632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd89eb-93c5-4f70-976e-5e31602e7669 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:14.236009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 652/1302] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:20:14.240999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:14.241415 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:14.241528 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:14.259593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:20:14.261387 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:20:14.324582 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 651/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:20:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 03:20:14.586323 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd89eb-93c5-4f70-976e-5e31602e7669 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:14.679962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:14.687120 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd89eb-93c5-4f70-976e-5e31602e7669 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:14.719977 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:14.720590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:14.720874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:14.721148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:14.721718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:14.723049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 03:20:14.731645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 03:20:14.732777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 04 03:20:14.733614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 653/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 03:20:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 03:20:14.738715 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:14.812833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:14.835004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb75b88-6a3e-4a53-ac83-3c0e81a31934 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:14.868010 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:14.868546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:14.868823 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:14.869092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:14.869707 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:20:14.870009 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:14.871392 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2a5c1-d949-4e52-9c1f-c14b3ec49756 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:14.876323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 652/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 03:20:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:20:14.881954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:20:14.976826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7292c8-7326-455e-962c-de094e8afd02 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:20:15.008964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:20:15.009546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:20:15.009818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:20:15.010087 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:20:15.010764 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:20:15.012130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c00682-7eef-4907-9b04-106205bafba0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:20:15.030966 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 654/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:20:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:20:24.873814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=30 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:20:25.027031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:19.018351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22240-4f83-414b-932b-85de0e13e66d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:19.018802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22240-4f83-414b-932b-85de0e13e66d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:19.019035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22240-4f83-414b-932b-85de0e13e66d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:19.020498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 653/1307] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:19.026874 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5e49-5b47-44b8-a66b-c4d0ab2cf107 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:19.027382 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5e49-5b47-44b8-a66b-c4d0ab2cf107 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:19.027583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5e49-5b47-44b8-a66b-c4d0ab2cf107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:19.193499 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-06ae5e49-5b47-44b8-a66b-c4d0ab2cf107 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:19.194703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-06ae5e49-5b47-44b8-a66b-c4d0ab2cf107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:19.388069 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 655/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:19.678476 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006114e7-c939-479a-b0cd-c23178541424 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:19.678814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006114e7-c939-479a-b0cd-c23178541424 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:19.679043 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006114e7-c939-479a-b0cd-c23178541424 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:19.680744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 654/1309] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 4 03:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:19.687304 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbacbeb-47eb-42e4-8a9d-7459ff415b42 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:19.687608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbacbeb-47eb-42e4-8a9d-7459ff415b42 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:19.687887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbacbeb-47eb-42e4-8a9d-7459ff415b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:19.776136 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6fbacbeb-47eb-42e4-8a9d-7459ff415b42 None None] MFA Rules not processed for user `dbf2caf4d02d44f19080cdc94f95ab4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:19.778108 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6fbacbeb-47eb-42e4-8a9d-7459ff415b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:19.800990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:19.872154 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 656/1310] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 4 03:21:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:19.873257 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006114e7-c939-479a-b0cd-c23178541424 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:19.875090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-006114e7-c939-479a-b0cd-c23178541424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:20.000393 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006114e7-c939-479a-b0cd-c23178541424 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:20.081841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:20.083300 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:20.083828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:20.084329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:20.086767 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:20.148575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:20.209443 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:20.212089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def0c7d-c410-4e1c-b1ef-d7e153cf6370 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:20.245306 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 655/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 03:21:22.692849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:22.693405 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:22.693502 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:22.696404 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 657/1312] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:22.701802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f28a2-a21b-44c4-b2e6-51237dde5e53 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:22.702224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f28a2-a21b-44c4-b2e6-51237dde5e53 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:22.702342 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f28a2-a21b-44c4-b2e6-51237dde5e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:22.771857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-277f28a2-a21b-44c4-b2e6-51237dde5e53 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:22.773650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-277f28a2-a21b-44c4-b2e6-51237dde5e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:22.868231 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 656/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:21:23.147992 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:23.149554 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:23.250619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe5059-f2c9-460c-9277-f29eb469b71d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:23.321500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:23.324161 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:23.324948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:23.325393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:23.325860 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:23.327476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:23.436324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:23.437681 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c51d89-1905-4859-930c-83247c258d27 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:23.454982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 658/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:25.877442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:25.877968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:25.878092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:25.879370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 657/1315] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:25.885097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bb40-2e64-47b8-8ac3-16dcb46070a6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:25.885511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bb40-2e64-47b8-8ac3-16dcb46070a6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:25.885641 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bb40-2e64-47b8-8ac3-16dcb46070a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:25.956576 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c866bb40-2e64-47b8-8ac3-16dcb46070a6 None None] MFA Rules not processed for user `1b9cf77b90164c2b900d5824c6606349`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:25.957787 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c866bb40-2e64-47b8-8ac3-16dcb46070a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:26.008211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:26.052050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 659/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:26.338930 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:26.340504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:26.350773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:26.445102 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56985b4-47f4-4300-92a0-580868c6c02a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:26.517089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:26.517688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:26.517967 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:26.518236 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:26.519853 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:26.580279 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:26.625504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:26.626869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b68a52-449a-4c56-8e5c-3d4ae59723a1 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:26.644539 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 658/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:29.300788 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:29.301324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:29.301324 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:29.302680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 660/1318] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:29.308459 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907e245-7198-4261-b01d-995032467858 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:29.308856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907e245-7198-4261-b01d-995032467858 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:29.309019 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907e245-7198-4261-b01d-995032467858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:29.374268 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4907e245-7198-4261-b01d-995032467858 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:29.375567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4907e245-7198-4261-b01d-995032467858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:29.470517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 659/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:29.757079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:29.758630 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:29.858903 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4deff0-d511-423a-8783-a5497a638739 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:29.934195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:29.934795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:29.935074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:29.935493 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:29.937079 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:29.944436 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:30.045241 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:30.046847 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fbab1-e64f-4ff0-a660-8389278511aa service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:30.065147 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 661/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:32.855215 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc1ce-1d73-4ea7-a5a7-38f7cf164809 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:32.855750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc1ce-1d73-4ea7-a5a7-38f7cf164809 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:32.855750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc1ce-1d73-4ea7-a5a7-38f7cf164809 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:32.857094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 660/1321] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:32.862275 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98391c-6bf5-4124-9ee1-ded1d1e2849c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:32.862695 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98391c-6bf5-4124-9ee1-ded1d1e2849c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:32.862964 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98391c-6bf5-4124-9ee1-ded1d1e2849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:32.936604 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0b98391c-6bf5-4124-9ee1-ded1d1e2849c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:32.937828 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0b98391c-6bf5-4124-9ee1-ded1d1e2849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:33.000328 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:33.042280 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 662/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:33.340404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:33.340688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:33.340967 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:33.342308 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 661/1323] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 4 03:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:33.348251 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2d75-7d44-407b-95c8-fe54820a6260 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:33.348538 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2d75-7d44-407b-95c8-fe54820a6260 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:33.348814 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2d75-7d44-407b-95c8-fe54820a6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:33.428084 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-19cb2d75-7d44-407b-95c8-fe54820a6260 None None] MFA Rules not processed for user `dbf2caf4d02d44f19080cdc94f95ab4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:33.429279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-19cb2d75-7d44-407b-95c8-fe54820a6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:33.503713 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 663/1324] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 4 03:21:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:33.513326 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:33.514886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:33.534999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:33.620934 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8543ad-ae87-49c6-94fe-679b1208441c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:33.692821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:33.693987 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:33.694260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:33.694524 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:33.696126 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:33.766923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:33.804009 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:33.805469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e7195-32fc-40bd-b8f1-e27842e6f98b service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:33.823372 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 662/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:36.264500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:36.265009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:36.265009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:36.266279 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 664/1326] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:36.271633 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a708be-bb07-4c12-bf2a-a8fc8e5f1078 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:36.272084 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a708be-bb07-4c12-bf2a-a8fc8e5f1078 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:36.272179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a708be-bb07-4c12-bf2a-a8fc8e5f1078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:36.342592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e1a708be-bb07-4c12-bf2a-a8fc8e5f1078 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:36.343908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e1a708be-bb07-4c12-bf2a-a8fc8e5f1078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:36.456560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 663/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:36.742751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:36.744367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:36.816023 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:36.867784 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88965e1-b4a5-4221-908c-751bb52cab21 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:36.944731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:36.945341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:36.945619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:36.945892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:36.947572 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:37.060660 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:37.063266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:37.065321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a4ff5-64e3-4bed-ba6c-9f4815e6dffc service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:37.084886 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 665/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:39.314700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:39.315239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:39.315357 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:39.316624 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 664/1329] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:39.322638 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e43b2-632c-495b-8dd4-c756064a7f50 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:39.323031 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e43b2-632c-495b-8dd4-c756064a7f50 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:39.323268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e43b2-632c-495b-8dd4-c756064a7f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:39.391826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6b6e43b2-632c-495b-8dd4-c756064a7f50 None None] MFA Rules not processed for user `1b9cf77b90164c2b900d5824c6606349`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:39.392982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6b6e43b2-632c-495b-8dd4-c756064a7f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:39.488148 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 666/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:21:39.769869 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:39.771965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:39.805481 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:39.880399 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee41db-4c3f-4d03-8f78-7ac6d4a742bd None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:39.958006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:39.959227 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:39.959510 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:39.959789 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:39.961413 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:40.042314 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:40.081355 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:40.082896 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13480-c981-4f6d-b7ca-3b3506248788 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:40.105569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 665/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:42.635960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:42.636542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:42.636542 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:42.637763 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 667/1332] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:42.643133 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3392b-dccc-4920-a791-7df4bb22f6f5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:42.643584 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3392b-dccc-4920-a791-7df4bb22f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:42.643712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3392b-dccc-4920-a791-7df4bb22f6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:42.709409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b9b3392b-dccc-4920-a791-7df4bb22f6f5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:42.710614 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9b3392b-dccc-4920-a791-7df4bb22f6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:42.808619 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 666/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:43.090650 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:43.092226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:43.161941 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:43.196132 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d6072-9675-4929-889d-7eb974aa08b9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:43.268646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:43.269248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:43.269528 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:43.269797 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:43.271507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:43.396781 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:43.398202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464863-dc5b-4826-aba2-e1d44b670f81 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:43.407987 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:43.416837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 668/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:45.580964 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:45.581517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:45.581517 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:45.583761 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 667/1335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:45.589197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d683-7777-49a5-ad8f-26e24d5fd7c0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:45.589629 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d683-7777-49a5-ad8f-26e24d5fd7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:45.589728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d683-7777-49a5-ad8f-26e24d5fd7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:45.655526 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ce52d683-7777-49a5-ad8f-26e24d5fd7c0 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:45.656647 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ce52d683-7777-49a5-ad8f-26e24d5fd7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:45.751091 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 669/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:46.035456 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:46.037525 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:46.077365 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:46.143907 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ba74-9b60-4d35-8cc2-7e0de9ee453c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:46.216225 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:46.216832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:46.217104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:46.217370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:46.218956 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:46.299437 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:46.325150 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:46.326612 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f692b-9a48-4c1f-8f5a-7eb06a783d6c service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:46.344814 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 668/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:48.781777 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:48.782308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:48.782308 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:48.783602 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 670/1338] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:48.788771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63eec3-300d-4fdb-b104-c8a97433bdc5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:48.789183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63eec3-300d-4fdb-b104-c8a97433bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:48.789324 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63eec3-300d-4fdb-b104-c8a97433bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:48.856916 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1b63eec3-300d-4fdb-b104-c8a97433bdc5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:48.858113 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1b63eec3-300d-4fdb-b104-c8a97433bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:48.952327 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 669/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:49.233195 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:49.234738 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:49.336889 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:49.365045 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e80035-84cd-4b2e-9f17-aa758f1f6eb7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:49.440191 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:49.440804 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:49.441080 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:49.441379 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:49.443667 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:49.550130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:49.552021 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcaaf01-dae0-414c-ae59-8d00f569436c service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:49.570393 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 671/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:53.638913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:53.639469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:53.639469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:53.640908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 670/1341] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:53.646300 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078809-a812-407d-a7ae-acc3881c2652 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:53.646690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078809-a812-407d-a7ae-acc3881c2652 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:53.646851 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078809-a812-407d-a7ae-acc3881c2652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:53.652463 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:53.720881 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0c078809-a812-407d-a7ae-acc3881c2652 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:53.722012 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0c078809-a812-407d-a7ae-acc3881c2652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:53.817833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 672/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:54.102378 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:54.103953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:54.153778 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:54.211605 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fc0e3-8c58-4130-bb61-1f6f397a05b1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:54.286104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:54.286722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:54.287005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:54.287317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:54.288913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:54.390708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:54.407867 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:54.409328 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969f000-d8d2-4d6b-918c-b4fdda7131e1 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:54.428276 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 671/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:21:57.202920 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:57.203524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:57.203524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:57.204839 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 673/1344] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:21:57.210033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60add-305a-47e1-a06c-2d7ff6574163 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:57.210441 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60add-305a-47e1-a06c-2d7ff6574163 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:57.210596 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60add-305a-47e1-a06c-2d7ff6574163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:57.276544 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d3b60add-305a-47e1-a06c-2d7ff6574163 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:21:57.277748 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d3b60add-305a-47e1-a06c-2d7ff6574163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:57.380144 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 672/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:21:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:21:57.664419 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:21:57.666007 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:57.756146 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:21:57.773437 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42249be-b5d5-490d-84cb-fe3146e2efaa None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:21:57.846061 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:21:57.846672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:21:57.846956 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:21:57.847266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:21:57.849472 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:21:57.961102 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:21:57.962539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26413297-e4be-4229-a2ed-5f6a80bf28af service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:21:57.980053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 674/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:21:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:02.062141 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:02.062608 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:02.062700 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:02.064058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 673/1347] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:02.069286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299f2b-85a4-4dc9-9de3-8a23ddeadcae None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:02.069718 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299f2b-85a4-4dc9-9de3-8a23ddeadcae None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:02.069833 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299f2b-85a4-4dc9-9de3-8a23ddeadcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:02.085902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:02.137838 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b4299f2b-85a4-4dc9-9de3-8a23ddeadcae None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:02.139104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b4299f2b-85a4-4dc9-9de3-8a23ddeadcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:02.240599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 675/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:02.532003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:02.533541 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:02.591806 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:02.637409 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585a3ed-742d-42e8-a783-3422f8bfa914 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:02.727008 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:02.727656 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:02.727937 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:02.728206 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:02.729902 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:02.841325 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:02.845749 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:02.847238 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b497c-5090-47be-98f1-f64eaee05d08 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:02.865556 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 674/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:07.754551 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4d12e-d32a-4867-9d43-8caaf238567a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:07.765651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4d12e-d32a-4867-9d43-8caaf238567a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:07.765651 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4d12e-d32a-4867-9d43-8caaf238567a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:07.767244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 676/1350] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 03:22:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:07.774885 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282918-ed7d-4a13-ab0a-a606d7352761 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:07.779006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282918-ed7d-4a13-ab0a-a606d7352761 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:07.779006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282918-ed7d-4a13-ab0a-a606d7352761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:07.929760 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e0282918-ed7d-4a13-ab0a-a606d7352761 None None] MFA Rules not processed for user `b50de3f398ee4cd487229cecf88a5b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:07.931136 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e0282918-ed7d-4a13-ab0a-a606d7352761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:08.057500 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 675/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 03:22:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:12.201742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497587-b80a-48fd-8522-236a37a49ad7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:12.202345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497587-b80a-48fd-8522-236a37a49ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:12.202345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497587-b80a-48fd-8522-236a37a49ad7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:12.203899 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 677/1352] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:12.208777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a9773-8ff1-412d-8eba-a3f8cdf2c354 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:12.209217 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a9773-8ff1-412d-8eba-a3f8cdf2c354 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:12.209315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a9773-8ff1-412d-8eba-a3f8cdf2c354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:12.238294 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=59 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:12.266676 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:12.291868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-df0a9773-8ff1-412d-8eba-a3f8cdf2c354 None None] MFA Rules not processed for user `b50de3f398ee4cd487229cecf88a5b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:12.293062 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-df0a9773-8ff1-412d-8eba-a3f8cdf2c354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:12.408066 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 676/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 03:22:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:13.669088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb1cc-fbd2-49c6-a92c-b3929828db29 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:13.669875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb1cc-fbd2-49c6-a92c-b3929828db29 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:13.669875 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb1cc-fbd2-49c6-a92c-b3929828db29 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:13.671118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 678/1354] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 03:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:13.676104 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832b3c-8214-4e3d-b23f-4a53904418e7 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:13.676494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832b3c-8214-4e3d-b23f-4a53904418e7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:13.676662 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832b3c-8214-4e3d-b23f-4a53904418e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:13.752752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-26832b3c-8214-4e3d-b23f-4a53904418e7 None None] MFA Rules not processed for user `b50de3f398ee4cd487229cecf88a5b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:13.754494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-26832b3c-8214-4e3d-b23f-4a53904418e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:13.796144 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:13.861160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 677/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 03:22:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:17.559799 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc3509-beb0-410f-ae7a-05b1dd55f6e7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:17.560346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc3509-beb0-410f-ae7a-05b1dd55f6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:17.560346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc3509-beb0-410f-ae7a-05b1dd55f6e7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:17.561905 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 679/1356] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:17.567193 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64b4d-cf4e-4881-996b-81e8087544cc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:17.567972 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64b4d-cf4e-4881-996b-81e8087544cc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:17.568275 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64b4d-cf4e-4881-996b-81e8087544cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:17.638009 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-88f64b4d-cf4e-4881-996b-81e8087544cc None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:17.639286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-88f64b4d-cf4e-4881-996b-81e8087544cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:17.753717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 678/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:18.056096 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:18.057225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:18.057715 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:18.059345 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 680/1358] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 03:22:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:18.067464 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6338bd-4e91-484d-a342-c60fef56eb62 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:18.067821 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6338bd-4e91-484d-a342-c60fef56eb62 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:18.068118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6338bd-4e91-484d-a342-c60fef56eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:18.086140 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:18.164400 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5b6338bd-4e91-484d-a342-c60fef56eb62 None None] MFA Rules not processed for user `2c5eae01c70b440e95b480d51780dd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:18.165591 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5b6338bd-4e91-484d-a342-c60fef56eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:18.291282 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 679/1359] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 03:22:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:18.299434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:18.301247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:18.461259 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955e95f-63c8-4ec2-81a9-efdc981c5f63 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:18.545287 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:18.589332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:18.590026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:18.590247 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:18.590485 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:18.592636 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:18.699029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:18.700655 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf7ad1-d3e7-464f-8f3b-414cf81f0a7b service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:18.719808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 681/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 03:22:21.110319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:21.110848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:21.110848 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:21.112205 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 680/1361] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:21.117806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0112-777e-4ab9-bf18-599ea261e695 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:21.118367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0112-777e-4ab9-bf18-599ea261e695 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:21.118367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0112-777e-4ab9-bf18-599ea261e695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:21.191420 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-754f0112-777e-4ab9-bf18-599ea261e695 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:21.192661 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-754f0112-777e-4ab9-bf18-599ea261e695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:21.232546 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:21.296704 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 682/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:21.584082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:21.585834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:21.654831 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:21.760936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f8ea9-7b58-44b6-8e7d-e8e2a6677315 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:21.897174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:21.897717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:21.898120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:21.898397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:21.900099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:21.923790 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:21.995220 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:21.996839 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced9edd-b9ed-4ab7-89fb-22d14efb1738 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:22.015362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 681/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:24.517167 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6a5c-2d2e-43c6-8f7f-b63745737c95 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:24.517726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6a5c-2d2e-43c6-8f7f-b63745737c95 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:24.517726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6a5c-2d2e-43c6-8f7f-b63745737c95 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:24.519685 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 683/1364] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:24.524999 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06117e-5c74-423b-8674-73722550760d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:24.525418 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06117e-5c74-423b-8674-73722550760d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:24.525565 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06117e-5c74-423b-8674-73722550760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:24.598421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-dc06117e-5c74-423b-8674-73722550760d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:24.600059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-dc06117e-5c74-423b-8674-73722550760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:24.709771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:24.712170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 682/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:25.001990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:25.002922 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:25.003343 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:25.005009 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 684/1366] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 03:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:25.010765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e15a-b6d2-484a-9101-7776306f945d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:25.011403 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e15a-b6d2-484a-9101-7776306f945d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:25.011833 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e15a-b6d2-484a-9101-7776306f945d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:25.093099 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9ab7e15a-b6d2-484a-9101-7776306f945d None None] MFA Rules not processed for user `2c5eae01c70b440e95b480d51780dd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:25.094468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9ab7e15a-b6d2-484a-9101-7776306f945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:25.217602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 683/1367] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 03:22:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:25.226281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:25.227944 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:25.387652 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7fe57-5f04-4bf5-bc2b-9171db8a7f7a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:25.409837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:25.516486 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:25.517262 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:25.517836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:25.517836 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:25.520156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:25.628101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:25.629712 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05d61-dac2-4b9b-a43d-2d1038eabab4 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:25.648479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 685/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 03:22:25.709773 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc48b-0409-4c82-9bd3-b17075a92415 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:25.710274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc48b-0409-4c82-9bd3-b17075a92415 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:25.710359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc48b-0409-4c82-9bd3-b17075a92415 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:25.715688 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 684/1369] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:25.717338 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d5bd7-642a-4aa1-a09d-23907c2a99c2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:25.717635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d5bd7-642a-4aa1-a09d-23907c2a99c2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:25.717931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d5bd7-642a-4aa1-a09d-23907c2a99c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:25.756292 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:25.808311 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1e5d5bd7-642a-4aa1-a09d-23907c2a99c2 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:25.810057 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1e5d5bd7-642a-4aa1-a09d-23907c2a99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:25.943202 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 686/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:26.276261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:26.276261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:26.276261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:26.280731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 685/1371] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 4 03:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:22:26.285255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f19f-3b39-4ab5-96e4-836627d2cecc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:26.285255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f19f-3b39-4ab5-96e4-836627d2cecc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:26.285255 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f19f-3b39-4ab5-96e4-836627d2cecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:26.362962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8ce4f19f-3b39-4ab5-96e4-836627d2cecc None None] MFA Rules not processed for user `447877c159084ece9e07080d0785e5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:26.364218 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8ce4f19f-3b39-4ab5-96e4-836627d2cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:26.419915 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:26.452074 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 687/1372] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 4 03:22:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:26.460254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:26.461801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:26.521077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:26.579534 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938846ee-ba94-411b-be3c-e880a97e7dfb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:26.664160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '447877c159084ece9e07080d0785e5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:26.664768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:26.665046 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:26.665319 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:26.666905 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:26.780878 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:26.796787 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:26.798446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fe472-2615-4953-8f58-aad9b487b575 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:26.820032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 686/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:27.902865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:27.903434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:27.903434 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:27.904770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 688/1374] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:27.910485 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574681a-efcf-426e-be6a-18c874c78e6d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:27.910958 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574681a-efcf-426e-be6a-18c874c78e6d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:27.911082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574681a-efcf-426e-be6a-18c874c78e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:27.976122 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a574681a-efcf-426e-be6a-18c874c78e6d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:27.977315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a574681a-efcf-426e-be6a-18c874c78e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:28.071552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 687/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:28.361058 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:28.363302 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:28.520158 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9f78-f406-431b-9f54-e141434ac247 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:28.573543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:28.635690 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:28.636332 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:28.636610 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:28.636877 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:28.638496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:28.736458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:28.738088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c9d2e-c0da-4d92-ab79-9ce490e56361 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:28.756758 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 689/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:30.960708 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:30.961265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:30.961265 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:30.962788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 688/1377] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:30.968543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759126-82a5-44dc-842a-b641a9738451 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:30.968959 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759126-82a5-44dc-842a-b641a9738451 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:30.969318 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759126-82a5-44dc-842a-b641a9738451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:31.037395 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1a759126-82a5-44dc-842a-b641a9738451 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:31.038780 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1a759126-82a5-44dc-842a-b641a9738451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:31.046328 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:31.150504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 690/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:31.442446 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:31.444194 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:31.508739 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:31.597574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc95e17-2250-4438-87a6-aaf3d825a47d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:31.714840 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:31.715553 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:31.715939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:31.716315 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:31.718002 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:31.738343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:31.815273 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:31.817178 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeff39-bd66-41d1-9386-9c4071041f16 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:31.835811 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 689/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:34.055590 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:34.056014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:34.056014 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:34.057355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 691/1380] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:34.062616 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343cba-1421-4d7c-9dc8-4365f43a828c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:34.063037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343cba-1421-4d7c-9dc8-4365f43a828c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:34.063335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343cba-1421-4d7c-9dc8-4365f43a828c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:34.141828 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f1343cba-1421-4d7c-9dc8-4365f43a828c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:34.143655 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f1343cba-1421-4d7c-9dc8-4365f43a828c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:34.269374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:34.275780 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 690/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:34.618409 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:34.620027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:34.789637 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:34.803967 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eee8e6-a0a4-4e26-8ff6-f302849d9507 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:34.943518 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:34.944050 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:34.944329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:34.944599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:34.946193 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:35.041458 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:35.043063 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201061-36a3-4447-890b-5add47d2d66d service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:35.054133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:35.063244 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 692/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:37.288971 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:37.289560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:37.289866 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:37.291838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 691/1383] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:37.297504 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262af5-c0fc-489b-bbc9-6c9feb8c256b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:37.297936 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262af5-c0fc-489b-bbc9-6c9feb8c256b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:37.298114 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262af5-c0fc-489b-bbc9-6c9feb8c256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:37.365560 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-33262af5-c0fc-489b-bbc9-6c9feb8c256b None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:37.366760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-33262af5-c0fc-489b-bbc9-6c9feb8c256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:37.463742 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 693/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:37.753039 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:37.755054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:37.921737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5c07-45ed-413d-8612-bdfcc173ecd6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:37.990138 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:38.039061 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:38.039798 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:38.040089 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:38.040359 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:38.041991 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:38.144042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:38.145691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5187-cbd4-4b83-9ae6-caf78d106f73 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:38.164212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 692/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:40.335449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:40.336054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:40.336054 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:40.337346 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 694/1386] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:40.342623 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7806-90ed-4c03-a806-8623d08bdc70 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:40.343077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7806-90ed-4c03-a806-8623d08bdc70 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:40.343219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7806-90ed-4c03-a806-8623d08bdc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:40.414033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ccda7806-90ed-4c03-a806-8623d08bdc70 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:40.415254 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ccda7806-90ed-4c03-a806-8623d08bdc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:40.448261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:40.513394 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 693/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:40.812566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:40.814115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:40.894503 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:40.973032 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace627f9-148d-48af-852d-77e1e19db669 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:41.086672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:41.087320 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:41.087596 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:41.087865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:41.089460 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:41.127550 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:41.179357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:41.180955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e93a4a-7c9b-42bd-bed7-7ed15fd270b9 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:41.199101 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 695/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:42.380369 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31718cb-fb00-4fcd-ac6a-0a7d0d344f9d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:42.381005 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31718cb-fb00-4fcd-ac6a-0a7d0d344f9d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:42.381592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31718cb-fb00-4fcd-ac6a-0a7d0d344f9d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:42.382635 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 694/1389] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:42.388664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e22df6-8433-4659-b617-390d2e83ec2f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:42.389088 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e22df6-8433-4659-b617-390d2e83ec2f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:42.389312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e22df6-8433-4659-b617-390d2e83ec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:42.463782 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f4e22df6-8433-4659-b617-390d2e83ec2f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:42.465003 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f4e22df6-8433-4659-b617-390d2e83ec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:42.565821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 696/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:42.865032 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:42.865323 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:42.865602 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:42.866968 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 695/1391] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 4 03:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:42.872355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9ad-b9c4-443c-9c27-dba49d3ac0ed None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:42.872696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9ad-b9c4-443c-9c27-dba49d3ac0ed None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:42.872976 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9ad-b9c4-443c-9c27-dba49d3ac0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:42.888626 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:42.952407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9b63b9ad-b9c4-443c-9c27-dba49d3ac0ed None None] MFA Rules not processed for user `7a7d32acd9af405c90b0c21fa8a20fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:42.953613 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9b63b9ad-b9c4-443c-9c27-dba49d3ac0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:43.039461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 697/1392] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 4 03:22:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:43.045411 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:43.047200 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:43.157812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffee3a9-8aa8-4510-9540-92ba6afb1c11 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:43.219018 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:43.235681 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d32acd9af405c90b0c21fa8a20fb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:43.236299 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:43.236582 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:43.236860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:43.238496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:43.362816 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:43.364484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235a28f-ede1-41cc-963d-5587309a9fd6 service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:43.385810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 696/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:43.386849 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:43.387266 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:43.387614 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:43.388960 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 698/1394] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:22:43.394873 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431903-a639-4593-a9c0-e27336c38805 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:43.395262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431903-a639-4593-a9c0-e27336c38805 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:43.395568 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431903-a639-4593-a9c0-e27336c38805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:43.470717 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c0431903-a639-4593-a9c0-e27336c38805 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:43.472046 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c0431903-a639-4593-a9c0-e27336c38805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:43.513246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:43.579115 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 697/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:43.868103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:43.869599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:43.975627 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:44.027539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed194e2-3d08-4ed3-97d2-fc8b6b931198 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:44.140255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d146df4-dd4a-44ac-abe9-175dfc0281a3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:44.140657 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d146df4-dd4a-44ac-abe9-175dfc0281a3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:44.140937 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d146df4-dd4a-44ac-abe9-175dfc0281a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:44.155274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:44.155890 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:44.156168 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:44.156449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:44.158053 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:44.208569 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6d146df4-dd4a-44ac-abe9-175dfc0281a3 None None] MFA Rules not processed for user `36aa8a9229e84ff29445df661f752fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:44.209949 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6d146df4-dd4a-44ac-abe9-175dfc0281a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:44.233407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:44.255970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:44.257567 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5d408-2e34-4a70-bc3a-a846d9743535 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:44.275772 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 699/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:44.297530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 698/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 03:22:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 03:22:44.348617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:44.348617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:44.348617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:44.348617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 700/1398] 10.222.0.15 () {54 vars in 943 bytes} [Tue Oct 4 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:44.351721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6702-09df-42be-bbe4-843f9b81c215 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:44.352097 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6702-09df-42be-bbe4-843f9b81c215 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:44.352469 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6702-09df-42be-bbe4-843f9b81c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:44.400080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:44.450014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a77f6702-09df-42be-bbe4-843f9b81c215 None None] MFA Rules not processed for user `bb8d1e9116254e08849307aa3450df44`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:44.451284 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a77f6702-09df-42be-bbe4-843f9b81c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:44.561956 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 699/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Tue Oct 4 03:22:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:44.571709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:44.573291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:44.711310 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f538-8f05-40df-a994-e2d6e0329371 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:44.766370 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:44.818130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb8d1e9116254e08849307aa3450df44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:44.818648 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:44.818932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:44.819235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:44.820866 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:44.904335 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:44.905932 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ceffa-0abb-4de8-81cd-5e8c0bbc8c16 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:44.914025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 701/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 03:22:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 03:22:46.491543 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:46.491966 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:46.492082 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:46.493718 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 700/1401] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:46.498726 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc9e3f-ed58-4b0d-8f71-4d601b5571d0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:46.499190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc9e3f-ed58-4b0d-8f71-4d601b5571d0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:46.499342 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc9e3f-ed58-4b0d-8f71-4d601b5571d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:46.580234 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cccc9e3f-ed58-4b0d-8f71-4d601b5571d0 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:46.582624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cccc9e3f-ed58-4b0d-8f71-4d601b5571d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:46.628496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:46.691384 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 702/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:46.984426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:46.986139 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:47.085093 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:47.152515 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b579-206e-48b9-b911-a7ffc5ef6bff None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:47.279422 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:47.280037 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:47.280396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:47.280751 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:47.282617 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:47.342925 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:47.383345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:47.385129 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450caf-8c8f-40b6-a8ca-06eb2309e92e service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:47.405756 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 701/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:49.607253 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:49.607820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:49.607820 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:49.609367 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 703/1404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:49.614622 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30f6b-8e34-414b-a12f-a1a96361dfab None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:49.615055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30f6b-8e34-414b-a12f-a1a96361dfab None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:49.615281 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30f6b-8e34-414b-a12f-a1a96361dfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:49.687661 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f5f30f6b-8e34-414b-a12f-a1a96361dfab None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:49.689410 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f5f30f6b-8e34-414b-a12f-a1a96361dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:49.790844 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 702/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:50.075830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:50.077357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:50.253433 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2522f7a-8c50-4033-9f17-e8d1900247c2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:50.284276 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:50.378707 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:50.379353 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:50.379635 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:50.379912 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:50.381524 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:50.484478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:50.487133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34383aa-44ce-4df6-8e99-dc24987b2ef7 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:50.510739 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 704/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:22:52.716721 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:52.717980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:52.718340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:52.720938 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 703/1407] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:22:52.726583 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2278a3-9cdf-4549-b76b-05c31a03c064 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:52.727026 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2278a3-9cdf-4549-b76b-05c31a03c064 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:52.727123 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2278a3-9cdf-4549-b76b-05c31a03c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:52.766135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:52.800882 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9e2278a3-9cdf-4549-b76b-05c31a03c064 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:22:52.802115 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9e2278a3-9cdf-4549-b76b-05c31a03c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:52.906156 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 705/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:22:53.197904 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:22:53.199460 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:53.239862 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:53.368867 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360ee27-8b33-439c-a756-c151dba8ea5f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:22:53.492211 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:22:53.500303 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:22:53.501639 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:22:53.502092 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:22:53.502362 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:22:53.504006 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:22:53.605830 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:22:53.607590 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e5e1d-15db-4406-9d11-d1137c0ce38f service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:22:53.628105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 704/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:02.904152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=52 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:03.625414 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=48 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:07.660713 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f34-9b61-4202-a872-ae5853dbf433 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:07.661226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f34-9b61-4202-a872-ae5853dbf433 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:07.661226 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f34-9b61-4202-a872-ae5853dbf433 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:07.662559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 706/1410] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 4 03:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:07.667561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7c11-675e-4a77-8c92-ef486e3609a6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:07.667954 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7c11-675e-4a77-8c92-ef486e3609a6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:07.668120 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7c11-675e-4a77-8c92-ef486e3609a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:07.744711 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-34af7c11-675e-4a77-8c92-ef486e3609a6 None None] MFA Rules not processed for user `b50de3f398ee4cd487229cecf88a5b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:07.745920 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-34af7c11-675e-4a77-8c92-ef486e3609a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:07.849942 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 705/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 4 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:07.863991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2397f2-4fda-4f30-94a7-fd5e801b19a7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:07.864280 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2397f2-4fda-4f30-94a7-fd5e801b19a7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:07.864559 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2397f2-4fda-4f30-94a7-fd5e801b19a7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:07.865870 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 707/1412] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 4 03:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:07.870445 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a26b6a-93f3-43e6-9716-17f03b577656 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:07.870730 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a26b6a-93f3-43e6-9716-17f03b577656 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:07.871033 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a26b6a-93f3-43e6-9716-17f03b577656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:07.947965 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:07.950774 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-44a26b6a-93f3-43e6-9716-17f03b577656 None None] MFA Rules not processed for user `2c5eae01c70b440e95b480d51780dd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:07.952552 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-44a26b6a-93f3-43e6-9716-17f03b577656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:08.065671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 706/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 4 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:10.666642 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:10.667197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:10.667197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:10.668442 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:10.674090 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b164-a327-4001-8b65-e95d1f4578ca None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:10.674507 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b164-a327-4001-8b65-e95d1f4578ca None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:10.674642 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b164-a327-4001-8b65-e95d1f4578ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:10.739701 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-26c0b164-a327-4001-8b65-e95d1f4578ca None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:10.740898 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-26c0b164-a327-4001-8b65-e95d1f4578ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:10.796056 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:10.838713 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:11.125789 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:11.127398 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:11.278317 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d099-adf6-4a61-9961-36ac97a059aa None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:11.350487 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:11.391731 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:11.392341 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:11.392617 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:11.392885 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:11.394477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:11.482513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:11.484168 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1de1f5-7d6d-44a2-9e26-9fb41f58950f service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:11.502242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 709/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:12.657823 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:12.658451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:12.658451 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:12.659683 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:12.665830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae706da-3dcb-4575-aaa3-ae699d4c14ba None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:12.666274 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae706da-3dcb-4575-aaa3-ae699d4c14ba None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:12.666377 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae706da-3dcb-4575-aaa3-ae699d4c14ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:12.739509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dae706da-3dcb-4575-aaa3-ae699d4c14ba None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:12.740698 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dae706da-3dcb-4575-aaa3-ae699d4c14ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:12.767770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:12.837360 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:12.987384 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:12.988827 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:13.130791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0e760-f0cf-42fe-8e98-112b20c30d62 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:13.169765 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:13.241859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:13.242396 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:13.242675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:13.242945 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:13.244575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:13.332335 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:13.333929 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23412d1-322e-4faf-84f3-b58c0dd41f81 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:13.351795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 709/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:14.469921 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:14.470556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:14.470556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:14.471911 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:14.478074 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0d57-ab4f-42c9-8fb8-5ede6adc5b1a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:14.478498 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0d57-ab4f-42c9-8fb8-5ede6adc5b1a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:14.478631 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0d57-ab4f-42c9-8fb8-5ede6adc5b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:14.533067 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:14.555535 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a20c0d57-ab4f-42c9-8fb8-5ede6adc5b1a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:14.556737 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a20c0d57-ab4f-42c9-8fb8-5ede6adc5b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:14.651331 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:14.800724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:14.802242 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:14.945845 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107773d-950c-40a6-ab22-4ba465f5eb3a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:14.957990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:15.075025 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:15.075943 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:15.076359 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:15.076771 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:15.079333 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:15.173093 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:15.174672 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3aacf-101e-4cd3-beba-defb76c0704f service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:15.192972 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 712/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:16.296802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:16.297340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:16.297340 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:16.298795 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:16.305111 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d4a00-64b2-4373-bf69-ac61f5259e58 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:16.305494 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d4a00-64b2-4373-bf69-ac61f5259e58 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:16.305666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d4a00-64b2-4373-bf69-ac61f5259e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:16.325041 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:16.381036 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-550d4a00-64b2-4373-bf69-ac61f5259e58 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:16.382248 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-550d4a00-64b2-4373-bf69-ac61f5259e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:16.478745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:16.628758 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:16.630272 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:16.742990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:16.778696 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39934bcf-0baf-43a5-b6f4-a183e1c059e5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:16.886812 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:16.887360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:16.887640 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:16.887908 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:16.889502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:16.964731 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:16.978791 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:16.980431 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed59a3-a5af-42c0-b56d-236e6ec46154 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:17.003045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 712/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:18.108991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:18.109653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:18.109653 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:18.111418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:18.117585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df78-99b2-4465-bb64-01869da22c56 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:18.118036 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df78-99b2-4465-bb64-01869da22c56 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:18.118166 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df78-99b2-4465-bb64-01869da22c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:18.194669 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-40c5df78-99b2-4465-bb64-01869da22c56 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:18.195913 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-40c5df78-99b2-4465-bb64-01869da22c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:18.294834 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:18.444561 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:18.446076 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:18.529328 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:18.591000 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48abc-29d5-4d9c-a4fe-c51058958d6b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:18.702862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:18.703426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:18.703709 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:18.704013 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:18.705619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:18.753714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:18.794372 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:18.796002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658eb8-dea1-43e7-8077-42ac881860af service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:18.814744 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 715/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:19.867091 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99714106-72a4-4684-81db-b527069218df None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:19.867694 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99714106-72a4-4684-81db-b527069218df None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:19.867694 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99714106-72a4-4684-81db-b527069218df None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:19.869053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:19.875069 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd2c3-21e2-4772-8625-13653903bd2a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:19.875496 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd2c3-21e2-4772-8625-13653903bd2a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:19.876171 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd2c3-21e2-4772-8625-13653903bd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:19.951619 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-38ffd2c3-21e2-4772-8625-13653903bd2a None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:19.952827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-38ffd2c3-21e2-4772-8625-13653903bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:20.046314 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:20.195838 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99714106-72a4-4684-81db-b527069218df None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:20.197262 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-99714106-72a4-4684-81db-b527069218df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:20.244990 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:20.349197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99714106-72a4-4684-81db-b527069218df None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:20.459442 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:20.459977 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:20.460255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:20.460523 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:20.462105 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:20.468822 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:20.551610 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:20.553216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9920-5f4e-487f-8380-4caea8b7e145 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:20.571053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 715/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:21.613430 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:21.613980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:21.613980 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:21.615489 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:21.621847 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82865091-c516-4fcc-b9c5-20ebcdbfcc04 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:21.622296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82865091-c516-4fcc-b9c5-20ebcdbfcc04 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:21.622405 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82865091-c516-4fcc-b9c5-20ebcdbfcc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:21.700461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-82865091-c516-4fcc-b9c5-20ebcdbfcc04 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:21.701606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-82865091-c516-4fcc-b9c5-20ebcdbfcc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:21.768496 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:21.797221 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:21.948162 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:21.949669 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:21.967571 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:22.094500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2fa30-c7a2-4d4e-9bbd-13bbc803cd15 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:22.180729 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:22.206857 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:22.207507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:22.207790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:22.208066 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:22.209674 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:22.297220 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:22.299268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74469bfd-633b-4441-b762-4411071eb742 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:22.317155 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 718/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:23.374658 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:23.375214 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:23.375345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:23.376643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:23.385476 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42506c-030f-4f65-818b-63a198f4562d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:23.385902 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42506c-030f-4f65-818b-63a198f4562d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:23.386030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42506c-030f-4f65-818b-63a198f4562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:23.461892 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ee42506c-030f-4f65-818b-63a198f4562d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:23.463130 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ee42506c-030f-4f65-818b-63a198f4562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:23.505017 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:23.562620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:23.712674 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:23.714107 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:23.856886 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d2a4f-d8a9-4914-91a3-c9695540ae35 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:23.911345 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:23.968538 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:23.969080 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:23.969939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:23.970231 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:23.971856 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:24.059571 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:24.061164 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdcc65-4451-40a6-be2b-8d9d30241cea service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:24.078439 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 718/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:25.127029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:25.127599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:25.127599 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:25.129135 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 03:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:25.135255 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0611900-3a6d-4699-95af-78a163d40ec3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:25.135685 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0611900-3a6d-4699-95af-78a163d40ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:25.135810 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0611900-3a6d-4699-95af-78a163d40ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:25.210412 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f0611900-3a6d-4699-95af-78a163d40ec3 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:25.211636 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f0611900-3a6d-4699-95af-78a163d40ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:25.213675 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:25.306224 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 03:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:25.456199 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:25.457723 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:25.604586 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f54b29-3f6e-4fb5-a4ca-855bcd604650 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:25.630981 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:25.721948 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:25.722556 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:25.722837 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:25.723110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:25.724735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:25.813369 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:25.814969 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abdcd-5652-4a97-b20b-aa41f9f80802 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:25.832806 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 721/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:27.877969 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:27.878530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:27.878530 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:27.879978 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:27.885138 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28518b4-eca9-4b34-83c8-bf01f164fc99 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:27.885544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28518b4-eca9-4b34-83c8-bf01f164fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:27.885691 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28518b4-eca9-4b34-83c8-bf01f164fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:27.931490 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:27.958048 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a28518b4-eca9-4b34-83c8-bf01f164fc99 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:27.959299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a28518b4-eca9-4b34-83c8-bf01f164fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:28.052379 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:28.338951 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:28.340505 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:28.442045 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c955b6-9112-428a-b5e9-50706d49d292 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:28.474246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:28.514426 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:28.515027 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:28.515334 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:28.515606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:28.517712 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:28.625910 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:28.627294 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303148-119a-4a9f-8b96-7334fa0d0cfe service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:28.645546 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 721/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:30.958970 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:30.959523 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:30.959699 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:30.961027 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:30.966179 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716ff-dc23-4dfc-9d15-107cdfd16761 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:30.966561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716ff-dc23-4dfc-9d15-107cdfd16761 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:30.966733 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716ff-dc23-4dfc-9d15-107cdfd16761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:31.031152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-899716ff-dc23-4dfc-9d15-107cdfd16761 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:31.032404 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-899716ff-dc23-4dfc-9d15-107cdfd16761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:31.045077 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:31.128488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:31.411024 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:31.412580 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:31.515701 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ad61-427d-41ec-aea3-9fe5aec19eb7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:31.528190 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:31.589348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:31.589954 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:31.590235 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:31.590507 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:31.592139 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:31.699858 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:31.701211 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b7dca-2051-4944-b52a-7c2f18e7865b service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:31.719913 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 724/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:34.177769 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:34.178339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:34.178339 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:34.179727 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:34.184910 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984851-d183-478e-a724-4d009251be5f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:34.185305 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984851-d183-478e-a724-4d009251be5f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:34.185465 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984851-d183-478e-a724-4d009251be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:34.233668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:34.252426 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-79984851-d183-478e-a724-4d009251be5f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:34.253632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-79984851-d183-478e-a724-4d009251be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:34.347990 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:34.633048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:34.634618 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:34.736801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c7ccd-feab-4bc2-a49b-3bd57e8e5900 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:34.779542 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:34.814420 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:34.815023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:34.815338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:34.815613 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:34.817201 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:34.924786 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:34.926204 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34c473-2f07-4b56-b64c-4ec7fa20b26e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:34.943521 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 724/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:37.249578 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:37.250152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:37.250152 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:37.252029 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:37.257186 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd86d4-50e8-4120-98fc-93e3a5c6c09e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:37.257585 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd86d4-50e8-4120-98fc-93e3a5c6c09e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:37.257734 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd86d4-50e8-4120-98fc-93e3a5c6c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:37.324042 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ebdd86d4-50e8-4120-98fc-93e3a5c6c09e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:37.325286 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ebdd86d4-50e8-4120-98fc-93e3a5c6c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:37.345575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:37.420503 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:37.717624 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:37.719708 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:37.831268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28578b67-a200-43cd-a51f-5a1d5f34565e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:37.852741 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:37.909911 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:37.910527 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:37.910808 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:37.911085 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:37.912760 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:38.023172 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:38.024517 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e90d6-dea1-409b-a7b2-bca8c0c6c76e service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:38.042800 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 727/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:40.302802 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:40.303363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:40.303363 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:40.304735 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:40.310133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270d0a6-a69c-4cdd-9519-68d5200ce843 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:40.310530 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270d0a6-a69c-4cdd-9519-68d5200ce843 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:40.310689 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270d0a6-a69c-4cdd-9519-68d5200ce843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:40.375926 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:40.380122 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b270d0a6-a69c-4cdd-9519-68d5200ce843 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:40.382299 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b270d0a6-a69c-4cdd-9519-68d5200ce843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:40.622627 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:40.906884 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:40.908448 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:41.024447 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809f56f-7d30-4c0a-8654-2a1bc650c7e6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:41.078502 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:41.103094 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:41.103768 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:41.104048 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:41.104320 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:41.105939 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:41.212963 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:41.214361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990ac84-a400-49a1-bb57-ab77d75af91b service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:41.231654 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 727/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:43.436703 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:43.437312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:43.437312 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:43.438620 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:43.444052 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde2d-2c5d-4d0f-9d4c-92b1e760f927 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:43.444491 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde2d-2c5d-4d0f-9d4c-92b1e760f927 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:43.444598 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde2d-2c5d-4d0f-9d4c-92b1e760f927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:43.513219 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-111bde2d-2c5d-4d0f-9d4c-92b1e760f927 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:43.514468 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-111bde2d-2c5d-4d0f-9d4c-92b1e760f927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:43.545371 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:43.616174 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:43.903107 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:43.904711 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:44.006049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044a827-6d48-486d-b38a-584cca75cde5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:44.034229 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:44.079208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:44.079816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:44.080098 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:44.080656 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:44.082256 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:44.196500 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:44.197848 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8d59a-3505-4d9e-9cf7-5b0ddfc9e0bb service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:44.215929 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 730/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:46.407627 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:46.408070 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:46.408170 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:46.409527 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:46.414735 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3332c-df86-44af-9c96-e76bc36edc17 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:46.415263 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3332c-df86-44af-9c96-e76bc36edc17 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:46.415373 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3332c-df86-44af-9c96-e76bc36edc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:46.481566 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b1c3332c-df86-44af-9c96-e76bc36edc17 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:46.482778 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b1c3332c-df86-44af-9c96-e76bc36edc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:46.491597 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:46.579893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:46.860826 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:46.862360 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:46.964777 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9129824-fc48-499d-b722-ecf45b945b77 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:47.019841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:47.038059 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:47.038659 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:47.038940 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:47.039244 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:47.040832 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:47.148195 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:47.149557 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a59bc-6ac6-4e5f-8e0a-2151b40936a5 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:47.167823 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 730/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 03:23:49.336745 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:49.337281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:49.337281 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:49.338611 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:49.343952 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe7c7-7bcb-4efe-8cba-fba0a39245e5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:49.344382 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe7c7-7bcb-4efe-8cba-fba0a39245e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:49.344494 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe7c7-7bcb-4efe-8cba-fba0a39245e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:49.412047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-70fbe7c7-7bcb-4efe-8cba-fba0a39245e5 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:49.413259 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-70fbe7c7-7bcb-4efe-8cba-fba0a39245e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:49.568270 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:49.627455 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:49.907812 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:49.909407 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:50.012205 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26a4c-78cd-4cb1-b5c9-a936700a3863 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:50.050095 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:50.085908 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:50.086513 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:50.086790 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:50.087062 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:50.088678 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:50.198535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:50.199897 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3020e-f0a0-47d1-8b7e-37923274afeb service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:50.218795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 733/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:52.361197 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:52.361801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:52.361801 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:52.363118 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:52.368431 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc971-a7df-4a7c-adbf-cfcf2984e72e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:52.368869 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc971-a7df-4a7c-adbf-cfcf2984e72e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:52.369002 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc971-a7df-4a7c-adbf-cfcf2984e72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:52.434608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-957cc971-a7df-4a7c-adbf-cfcf2984e72e None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:52.435844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-957cc971-a7df-4a7c-adbf-cfcf2984e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:52.453947 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:52.532826 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:52.810923 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:52.812480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:52.912752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31ec5d-3474-4f25-9ef5-10374f4e5c78 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:52.978558 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:52.986004 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:52.986628 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:52.986889 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:52.987184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:52.988813 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:53.095130 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:53.096574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409260-faa3-429c-9566-3e12bf139614 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:53.114803 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 733/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:23:55.281863 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:55.282427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:55.282427 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:55.283803 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 03:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:23:55.289370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a0c5-febf-4c08-adbb-87402827fab1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:55.289744 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a0c5-febf-4c08-adbb-87402827fab1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:55.289909 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a0c5-febf-4c08-adbb-87402827fab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:55.356017 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f788a0c5-febf-4c08-adbb-87402827fab1 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:23:55.357222 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f788a0c5-febf-4c08-adbb-87402827fab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:55.405035 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:55.453377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 03:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:23:55.732694 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:23:55.734225 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:55.835955 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0c999-6e76-41c3-bc2b-5772694743a0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:23:55.884615 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:23:55.911326 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:23:55.911938 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:23:55.912222 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:23:55.912497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:23:55.914104 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:23:56.020865 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:23:56.022207 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fccdc-96b2-40de-a00b-52d85f9f2cc3 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:23:56.040007 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 736/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:05.451868 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=22 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:06.037805 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=66 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:15.938672 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a1cd6-8cfa-4765-b660-4d330b770aaf None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:15.939295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a1cd6-8cfa-4765-b660-4d330b770aaf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:15.939295 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a1cd6-8cfa-4765-b660-4d330b770aaf None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:15.940592 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 735/1471] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:15.945859 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3fd3-8709-4a3e-b0a0-a3823ff2b17c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:15.946277 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3fd3-8709-4a3e-b0a0-a3823ff2b17c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:15.946415 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3fd3-8709-4a3e-b0a0-a3823ff2b17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:16.011680 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e5ba3fd3-8709-4a3e-b0a0-a3823ff2b17c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:16.012887 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e5ba3fd3-8709-4a3e-b0a0-a3823ff2b17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:16.108425 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 737/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:16.408484 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:16.408766 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:16.409065 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:16.410560 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 736/1473] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 4 03:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:16.415380 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10613-1ca3-4212-8df0-4952acf248f9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:16.415668 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10613-1ca3-4212-8df0-4952acf248f9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:16.415939 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10613-1ca3-4212-8df0-4952acf248f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:16.490909 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7fb10613-1ca3-4212-8df0-4952acf248f9 None None] MFA Rules not processed for user `7a7d32acd9af405c90b0c21fa8a20fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:16.492140 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7fb10613-1ca3-4212-8df0-4952acf248f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:16.512217 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:16.567714 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 738/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 4 03:24:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:16.575461 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:16.576961 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:16.678383 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ad5bc-640f-4311-b43b-91ff236a6298 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:16.750449 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d32acd9af405c90b0c21fa8a20fb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:16.751054 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:16.751361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:16.751634 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:16.753216 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:16.806229 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:16.859800 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:16.861269 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5505aef-d0a9-414c-bca6-82e5d894e149 service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:16.879452 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 737/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:19.091795 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:19.092365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:19.092365 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:19.093664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 739/1476] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:19.098788 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409469-e8c6-47ff-ad8a-7e37d54a3250 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:19.099184 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409469-e8c6-47ff-ad8a-7e37d54a3250 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:19.099364 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409469-e8c6-47ff-ad8a-7e37d54a3250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:19.164149 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-da409469-e8c6-47ff-ad8a-7e37d54a3250 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:19.165329 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-da409469-e8c6-47ff-ad8a-7e37d54a3250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:19.265373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 738/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:19.534818 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:19.536376 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:19.658165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870470e1-abd6-43ab-908f-c3df1c6aa163 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:19.705891 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:19.750649 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:19.751286 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:19.751570 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:19.751842 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:19.752474 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:19.754197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:24:19.754197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 04 03:24:19.754197 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: Oct 04 03:24:19.755380 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eec981-3719-40b4-b8f9-6c6d6d3fa71c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:19.774110 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 740/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 03:24:19 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 03:24:21.935101 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:21.935691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:21.935691 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:21.937594 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 739/1479] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:21.942827 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddf8a-532b-4430-a38b-44825cb34f2f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:21.943309 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddf8a-532b-4430-a38b-44825cb34f2f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:21.943442 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddf8a-532b-4430-a38b-44825cb34f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:22.020165 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-286ddf8a-532b-4430-a38b-44825cb34f2f None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:22.021394 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-286ddf8a-532b-4430-a38b-44825cb34f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:22.114982 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 741/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:22.406003 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:22.407650 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:22.417260 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:22.551783 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242197-3eef-425f-81fb-1c1da008ebbf None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:22.630234 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:22.662740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c5eae01c70b440e95b480d51780dd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:22.663296 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:22.663574 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:22.663841 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:22.665417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:22.756537 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:22.758134 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381237a-b823-4ea8-b6a3-e6389aabb57f service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:22.775811 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 740/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:25.098940 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:25.099545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:25.099545 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:25.100825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 742/1482] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:25.106014 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724456f-5082-43d1-abcc-2b1dad5b116c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:25.106419 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724456f-5082-43d1-abcc-2b1dad5b116c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:25.106563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724456f-5082-43d1-abcc-2b1dad5b116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:25.173350 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1724456f-5082-43d1-abcc-2b1dad5b116c None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:25.174548 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1724456f-5082-43d1-abcc-2b1dad5b116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:25.220123 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:25.270373 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 741/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:25.557163 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:25.558728 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:25.582374 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:25.661183 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f30732-457f-42b9-ad2c-38c771c090a4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:25.731754 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:25.732355 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:25.732634 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:25.732904 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:25.735071 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:25.807618 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:25.842008 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:25.843984 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a5c56-3836-4fc5-9762-74e28d7b5d24 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:25.862133 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 743/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:28.052268 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:28.052829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:28.052829 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:28.054245 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 742/1485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:28.059478 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc191f9a-04f6-4616-9d34-39c0c15b7968 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:28.059923 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc191f9a-04f6-4616-9d34-39c0c15b7968 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:28.060030 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc191f9a-04f6-4616-9d34-39c0c15b7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:28.126535 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fc191f9a-04f6-4616-9d34-39c0c15b7968 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:28.127770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc191f9a-04f6-4616-9d34-39c0c15b7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:28.225822 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 744/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:28.514138 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:28.516212 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:28.619611 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc350-9816-478f-a1af-df7c9102b875 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:28.690561 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:28.692159 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:28.692992 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:28.693458 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:28.693771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:28.695719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:28.802351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:28.803804 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd7d31-6ff1-49d8-80a0-39dc43c89fb1 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:28.821719 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 743/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:31.154452 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:31.154974 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:31.154974 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:31.156291 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 745/1488] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:31.161488 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e8d-894e-4363-abe1-b62cd3959f49 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:31.161926 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e8d-894e-4363-abe1-b62cd3959f49 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:31.162026 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e8d-894e-4363-abe1-b62cd3959f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:31.227732 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6b6d1e8d-894e-4363-abe1-b62cd3959f49 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:31.228944 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6b6d1e8d-894e-4363-abe1-b62cd3959f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:31.281563 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:31.325578 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 744/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:31.613011 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:31.614564 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:31.650632 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:31.718268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e64ac-2e24-493d-bd33-c3d310ba91f6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:31.789664 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:31.790261 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:31.790543 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:31.790816 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:31.792418 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:31.870410 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:31.899654 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:31.901121 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14190e-a41e-4973-bdcf-8e5406d3cc9a service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:31.919901 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 746/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:34.259358 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:34.259936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:34.259936 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:34.261246 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 745/1491] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 03:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:24:34.267059 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bfe3-6928-46cb-9848-5c32b69b820b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:34.267477 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bfe3-6928-46cb-9848-5c32b69b820b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:34.267696 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bfe3-6928-46cb-9848-5c32b69b820b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:34.333749 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e6b0bfe3-6928-46cb-9848-5c32b69b820b None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:24:34.334962 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e6b0bfe3-6928-46cb-9848-5c32b69b820b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:34.432514 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 747/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 03:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:24:34.728338 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:24:34.729858 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:34.831361 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26ca31-8acf-4f9e-964e-c1c43bea943a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:24:34.903250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:24:34.903843 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:24:34.904124 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:24:34.904397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:24:34.906771 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:24:34.915261 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:35.015239 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:24:35.016670 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7b00-a507-4e0e-b272-310c1d7abaa2 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:24:35.034480 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 746/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:24:44.431006 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=82 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:24:45.032271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=50 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:53.871639 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:53.872212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:53.872212 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:53.946994 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:25:53.948243 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.042946 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 748/1494] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 03:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:25:54.051160 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:54.051643 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:54.051750 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:54.126597 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:25:54.127857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.222785 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 747/1495] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 03:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:25:54.227421 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:54.228645 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.239831 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b8de4-4f99-43c3-b4a8-705031f542f7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:54.300361 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:54.336730 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:54.337331 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:54.337607 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:54.337880 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:54.338509 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 03:25:54.338807 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:54.339844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:54.339844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 04 03:25:54.339844 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: Oct 04 03:25:54.340721 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:54.351621 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 749/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 03:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 03:25:54.359914 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:54.368980 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138e2fa-2d78-409a-90c2-e1cbf37c6e42 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:54.429855 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:54.463377 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:54.463975 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:54.464251 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:54.464520 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:54.465183 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:54.466903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:54.466903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 04 03:25:54.466903 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: Oct 04 03:25:54.468128 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:54.522677 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 748/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 03:25:54 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 03:25:54.526872 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:54.528931 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.562825 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca90bb-043e-449e-a954-c1a557b110d6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:54.672461 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:54.672991 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:54.673268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:54.673539 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:54.674181 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:54.675257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:54.675257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 04 03:25:54.675257 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: Oct 04 03:25:54.676497 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:54.719397 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 750/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 03:25:54 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 03:25:54.723531 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:54.725055 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.785047 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a3e7-2ace-4faa-a696-3431bea2f063 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:54.873370 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:54.884271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:54.884874 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:54.885152 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:54.885423 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:54.886058 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:54.887606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:54.887606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 04 03:25:54.887606 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: Oct 04 03:25:54.888490 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:54.900504 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 749/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 03:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:25:54.905329 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:54.906850 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:54.918118 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894c6ca-5faf-4223-bf2e-09da95a123e9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:54.943357 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:55.014001 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:55.014608 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.014884 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.015177 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.015821 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:55.016834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:55.016834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 04 03:25:55.016834 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: Oct 04 03:25:55.017724 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3db57a-67c6-4a37-ba3b-4f5f6eb9f3cd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:55.030268 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 751/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 03:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 03:25:55.034421 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:55.035953 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.069381 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c2429-17e5-4a80-9b73-d4e0e89c00b6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:55.160397 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:55.161051 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.161330 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.161603 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] PATH_INFO: `/v3/projects/8b9c643719454165a2c330f1a73cd6ee/users/4ee53eb1ab534636ac8bbf08948fa2a8/roles/800a18a269264faba7cbec11735ac79b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.185250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9c643719454165a2c330f1a73cd6ee, user_id=4ee53eb1ab534636ac8bbf08948fa2a8, role_id=800a18a269264faba7cbec11735ac79b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:55.186274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:55.186274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 04 03:25:55.186274 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: Oct 04 03:25:55.188233 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd33df-5108-4962-950b-95ced38471aa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:55.212391 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 750/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 03:25:55 2022] PUT /identity/v3/projects/8b9c643719454165a2c330f1a73cd6ee/users/4ee53eb1ab534636ac8bbf08948fa2a8/roles/800a18a269264faba7cbec11735ac79b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 03:25:55.224544 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca267bb-98b5-4ada-b263-1bb044ae05a4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.224830 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca267bb-98b5-4ada-b263-1bb044ae05a4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.225103 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca267bb-98b5-4ada-b263-1bb044ae05a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.281037 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dca267bb-98b5-4ada-b263-1bb044ae05a4 None None] MFA Rules not processed for user `4ee53eb1ab534636ac8bbf08948fa2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:25:55.282237 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dca267bb-98b5-4ada-b263-1bb044ae05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.359481 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 752/1502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 03:25:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:25:55.373417 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.373709 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.373985 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.375389 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 751/1503] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 4 03:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 03:25:55.380479 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a996-fe9a-4026-b0af-6260d311c884 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.381187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a996-fe9a-4026-b0af-6260d311c884 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.381187 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a996-fe9a-4026-b0af-6260d311c884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.411569 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:55.455893 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7d45a996-fe9a-4026-b0af-6260d311c884 None None] MFA Rules not processed for user `447877c159084ece9e07080d0785e5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:25:55.457097 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7d45a996-fe9a-4026-b0af-6260d311c884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.530432 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 753/1504] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 4 03:25:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:25:55.538317 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:55.539859 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.589457 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:55.641374 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241d098-936c-4bed-93ee-ad541bff5192 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:55.712722 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '447877c159084ece9e07080d0785e5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:55.713312 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.713589 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.713860 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.716250 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.810575 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:55.812023 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:55.815386 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:55.830646 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 752/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:25:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:25:55.876178 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d085b-e691-4ee9-a1ea-5b6e903037c4 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:55.876464 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d085b-e691-4ee9-a1ea-5b6e903037c4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:55.876737 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d085b-e691-4ee9-a1ea-5b6e903037c4 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:55.878034 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 754/1506] 10.222.0.15 () {52 vars in 830 bytes} [Tue Oct 4 03:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 03:25:55.881593 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:55.882793 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:55.979883 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b823a-0920-4529-9e8a-fc6dfd221966 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:56.071757 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:56.072351 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:56.072586 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:56.072857 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] PATH_INFO: `/v3/users/4ee53eb1ab534636ac8bbf08948fa2a8` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:56.085407 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee53eb1ab534636ac8bbf08948fa2a8)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:56.086427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:56.086427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 04 03:25:56.086427 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: Oct 04 03:25:56.087671 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:56.097168 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:56.152752 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee53eb1ab534636ac8bbf08948fa2a8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:56.153185 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee53eb1ab534636ac8bbf08948fa2a8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:56.153705 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee53eb1ab534636ac8bbf08948fa2a8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:56.166932 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee53eb1ab534636ac8bbf08948fa2a8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:56.178285 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee53eb1ab534636ac8bbf08948fa2a8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:56.207001 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 753/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 03:25:55 2022] DELETE /identity/v3/users/4ee53eb1ab534636ac8bbf08948fa2a8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:25:56.211666 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:56.212145 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:56.212256 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:56.269149 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] MFA Rules not processed for user `3a6f18cf4a5b4b16ab604dd0d0188854`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 03:25:56.270349 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:56.345321 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:56.358898 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 755/1508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 03:25:56.369567 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:56.370794 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:56.473343 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c9e-5375-4b69-9aa8-fc7573e5c61e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:56.526271 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:25:56.548588 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf2caf4d02d44f19080cdc94f95ab4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '719ab2e182354971bdbb56ff51aa6ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:56.549198 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:56.549475 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:56.549740 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:56.551353 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:56.661667 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:56.663096 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3fb1e-5557-4a94-b330-4dae8f7344a7 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:56.681333 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 754/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 03:25:57.037459 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 03:25:57.038704 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 03:25:57.050511 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079378a-41e8-4d9d-8f13-595c74ab252d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 03:25:57.150208 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6f18cf4a5b4b16ab604dd0d0188854', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c792b1b39084f70bab95471acabbdd7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 03:25:57.150770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 03:25:57.151049 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 03:25:57.151348 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] PATH_INFO: `/v3/projects/8b9c643719454165a2c330f1a73cd6ee` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 03:25:57.161751 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9c643719454165a2c330f1a73cd6ee)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 03:25:57.162770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 03:25:57.162770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 04 03:25:57.162770 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: Oct 04 03:25:57.164005 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 03:25:57.220862 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9c643719454165a2c330f1a73cd6ee', 'request_id': 'req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:57.230908 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9c643719454165a2c330f1a73cd6ee', 'request_id': 'req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:57.243646 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9c643719454165a2c330f1a73cd6ee', 'request_id': 'req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:57.266606 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] The token cache is being invalidate because project 8b9c643719454165a2c330f1a73cd6ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 03:25:57.266895 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:57.267449 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7b1b6755-741d-4c5e-9ad6-6f02bf27d1e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 03:25:57.268740 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 756/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 03:25:57 2022] DELETE /identity/v3/projects/8b9c643719454165a2c330f1a73cd6ee => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 03:26:06.679053 dv-cinder-849143-9-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=62 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 03:26:07.264609 dv-cinder-849143-9-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}